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Abstract 

In the first part, in the local non archimedean case, we consider 
distributions on GL{n + 1) which are invariant under the adjoint ac- 
tion of GL{n). We conjecture that such distributions are invariant by 
transposition. This would imply multiplicity at most one for rerstric- 
tions from GL{n + 1) to GL{n). We reduce ourselves to distributions 
with "singular" support and then finish the proof for n < 8. 
In the second part we show that similar Theorems for orthogonal or 
unitary groups follow from the case of GL{n) 



Introduction 

Let F be a local field non archimedean and of characteristic 0. Let W be 
a vector space over F of finite dimension + 1 ^ 2 and let = \^ © f/ be a 
direct sum decomposition with dim\^ = n. Then we have an imbedding of 
GL{y) into GL{W). Our goal is to study the following Conjecture: 

Conjecture 1: If vr (resp. p) is an irreducible admissible representation of 
GL{W) (resp. ofGL{V)) then 

dim (HomGL(y)(7r|GL(y) , p)) ^ 1 

We choose a basis of V and a non zero vector in U thus getting a basis of 
W. We can identify GL{W) with GL{n + 1,F) and GL{V) with GL{n,¥). 

^The first author is partially supported by NSF Grant DMS-0500392 



The transposition map is an involutive anti automorphism of GL(n + l.F) 
which leaves GL{n, F) stable. It acts on the space of distributions on GL{n + 
1,F). 

Conjecture 1 is a Corollary of : 

Conjecture 2: A distribution on GL{W) which is invariant under the ad- 
joint action of GL{V) is invariant by transposition. 

To try to prove this conjecture we proceed by induction on n. We use 
Harish-Chandra's descent and another type of descent, of a more elementary 
nature, similar to the argument used to get Probenius reciprocity. This allows 
us to show that an invariant distribution, skew with respect to transposition 
must be supported by the "singular set" ( strictly speaking wc first linearize 
the problem). We have not been able to produce a complete proof that such 
a distribution must be 0, except for very small values of n ( precisely n ^8). 
At this stage we can not rule out the possibility that Conjecture 2 is false in 
general, thus casting a doubt on Conjecture 1. 

On the opposite we are still confident that this does not happen and that 
the missing part of the proof should not lie too deep. 

One can raise a similar question for orthogonal and unitary groups. Let D 
be a either F or a quadratic extension of F . If x e D then x is the conjugate 
of a; if D 7^ F and is equal to a; if © = F. 

Let 1^ be a vector space over D of finite dimension n + 1 ^ 2. Let (., .) 
be a non degenerate hermitian form on W. This form is bi-additive and 

{dw, d'w') — d d'{w, w'), {w', w) — {w, w') 

Given a D— hnear map u from W into itself, its adjoint u* is defined by the 
usual formula 

{u{w),w') — {w,u*{w')) 

Choose a vector e inW such that (e, e) 7^ 0; let C/ = De and V — JJ-^ the 
orthogonal complement. Then V has dimension n and the restriction of the 
hermitian form to V is non degenerate. 

Let M be the unitary group of W that is to say the group of all D— linear 
maps m oiW into itself which preserve the hermitian form or equivalently 
such that mm* — 1. Let G be the unitary group of V. With the p-adic 
topology both groups are of type Ictd ( locally compact, totally discontinuous 
and countable at infinity). They are reductive groups of classical type. 
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The group G is naturally imbedded into M. 

Conjecture 1': If vr (resp p) is an irreducible admissible representation of 
M (resp of G) then 

dim (HomG(vr|M,p)) < 1 
Choose a basis ei, . . . e„ of such that (cj, Cj) G F. For 

n 

W = XqC + XiCi 
1 

put 

n 

w = xe + xi Ci 
1 

If is a D— linear map from W into itself, let u be defined by 

u{w) = u(w) 

Let a be the anti-involution cr(m) = rn~^ of M; Conjecture 1' is a conse- 
quence of 

Conjecture 2': A distribution on M which is invariant under the adjoint 
action of G is invariant under a. 

We proceed exactly as in the case of the general linear group. Note that 
the Levi subgroups of some parabolic subgroups of G have components of 
type GL so that we need to assume Conjecture 2. However, in this case, 
the singular set has a natural stratification stable by the involution and it 
is possible to use induction on each strata, thus finishing the proof that 
Conjecture 2' follows from Conjecture 2. 

The above questions are not new and Conjectures 1 and 1' have been 
around for some time. At the beginning stage of this project the first author 
had conversations with J.Bernstein who was interested in the same problem. 

Assuming multiplicity at most one, a more difficult question is to find 
when it is one. Some partial results are known. 

For the orthogonal group (in fact the special orthogonal group) this ques- 
tion has been studied by B. Gross and D.Prasad ([T],[8]) who formulated a 
precise conjecture. An up to date account is given by B. Gross and M.Reeder 
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([2]). Perhaps their approach could eventually give a proof of Conjecture 
1'. In a different setup, in their work on "Shintani" functions A.Murase and 
T.Sugano obtained complete results for GL{n) and the split orthogonal case 
but only for spherical representations ([5],[7j). Finally we should mention, 
Hakim's publication [3], which, at least for the discrete series, could perhaps 
lead to a proof of the Conjectures. 

This work is divided in two parts (General Linear Group and Orthog- 
nal/Unitary Groups), each having a short introduction and an Appendix 
where we present a certainly well known elementary descent method based 
on an argument a la Frobenius. 
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Part I 
General Linear Group 



1 . Introduction 

Let F be a local field non archimedean and of characteristic 0. Let be a 
vector space over F of finite dimension n + 1 ^ 2 and let W = V (B U he 
a direct sum decomposition with dim!/ = n. Then we have an imbedding 
of GL{V) into GL{W). Our ( at present unachieved) goal is to prove the 
following Conjecture: 

Conjecture 1 //tt (resp. p) is an irreducible admissible representation of 
GL{W) (resp. of GL{V)) then 

dim(HomGL(v)(7r|GL(v) , p)) < 1 

We choose a basis of V and a non zero vector in U thus getting a basis of W. 
We can identify GL{W) with GL{n + 1,F) and GL{V) with GL{n,¥). The 
transposition map is an involutive anti automorphism of GL{n + 1, F) which 
leaves GL{n, F) stable. It acts on the space of distributions on G'L(n + 1,F). 
Conjecture 1 is a Corollary of : 

Conjecture 2 A distribution on GL{W) which is invariant under the ad- 
joint action of GL{y) is invariant by transposition. 

Remarks. — The transposition depends on the choice of the basis. How- 
ever, changing the basis of V amounts to the adjoint action of some element 
of GL{y) and so does not change the action on G'L(y)— invariant distri- 
butions. Changing the choice of the non zero vector in U amounts to the 
adjoint action of some element of GL{U). However, because the adjoint ac- 
tion of the center of GL{W) is trivial the adjoint action of GL{U) reduces 
to the adjoint action of the center of GL{V) and so is trivial on the space 
of GL(1/)— invariant distributions. In other words a distribution invariant 
under the adjoint action of GL{V) is in fact invariant under the adjoint ac- 
tion of GL{V) X GL{U) and so, on this space of distributions, the action of 
the "transposition" is indepcndant the choice of the basis, adapted to the 
decomposition W = V ® U . 
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We now describe in more details the content of this part. In section 2 we 
prove that Conjecture 1 follows from Conjecture 2. In section 3 we perform 
a partial linearization. Namely we consider the action of G = GL{V) on 
G X V X V* with V* the dual space of V; the group G acts on itself by the 
adjoint action, on V by the natural action and on V* by the inverse of the 
transpose. If (cj) is a basis of V and (e*) the dual basis of V* define the 
linear map u from V to V* by u{ei) — e* and consider the involution 

a : ig,v,v*) ^ {u~^ ^g%u~^{v*),u{v)) 

We show that Conjecture 2 is implied by 

Conjecture 3 A distribution on G xV xV* , invariant under G is invariant 
under a. 

Using the adjoint action on the Lie algebra of G and defining a as before, 
we then consider 

Conjecture 4 A distribution on QxV xV* , invariant under G is invariant 
under a. 

In section 4 we prove Conjecture 4 for the case n — 1, thus starting the 
induction. In section 5 we review Harish-Chandra's descent and first prove 
that Conjecture 4 implies Conjecture 3. 

Using Harish-Chandra's descent on the Lie algebra we get that an invari- 
ant distribution T on q x V x V* , skew-symmetric with respect to a, has a 
support contained in {Z x J\f) x V x V* where Z is the center of g and 
the cone of nilpotent elements. 

The next step is to study the support on the V Q) V* side. In section 
6 we define and study the regular orbits. A triplet (X, v, v*) is regular if 
(v, Xv, . . . , x"~^v) is a basis of V and if {v*,^ Xv*, . . . ,* X"'~'^v*) is a basis of 
V*. The set of regular elements is a non empty Zariski open subset, stable 
by G. The orbits of the regular elements are the "regular" orbits. They turn 
out to be closed, with trivial centralizer and stable by a. A well known result 
of Gelfand and Kazhdan then tells us that an invariant distribution on the 
set of regular elements is stable by a. 

A triplet {X,v,v*) is "singular" ii X e Af and if, for all i we have 
{v*,X'^v) — 0. In section 7 we show that if the triplet is non singular then 
the situation may be reduced to a mixture of the regular case and a case of 
lower dimension. We then can conclude that an invariant skew symmetric 
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distribution must have a support contained in the singular set ( note that Z 
does not play any role). 

In the remainder of this first Part we explain how far we can go to- 
ward proving that an invariant distribution with singular support and skew 
symmetric is 0. First in section 8 we collect further properties of these dis- 
tributions. They are distributions on [0,0] y.V y.V* . Define the Fourier 
transform on [g, g] using the Killing form. The Fourier transform of the dis- 
tribution has the same invariance properties. In particular its support must 
be singular. Corresponding to the Killing form there is a representation of 
the twofold covering of SL{2) and the distribution is invariant. For n even this 
implies that the distribution is and for n odd that it is homogeneous and 
equal to its Fourier transform. A similar remark is valid for the quadratic 
form {v*,v) on V (BV*. We fix a nilpotent orbit and transfer the problem to 
V®V*. 

In sections 9 and 10, working on V ® V* , we have a fix nilpotent matrix 
X and the distribution is invariant under the centrahzer C oi X. We try to 
work by induction on n( independantly of our general induction) but "going 
down" we get only invariance under a subgroup of the centrahzer of the new 
nilpotent matrix. This kills the hope of an easy proof. Still it works in a 
limited number of cases and we managed to finish the proof for n < 8. 

Finally section 11 outlines a completely different approach. We show that 
it would be enough to establish a symmetry property either of C-invariant 
linear forms on the space of a finite number of representations belonging to a 
degenerate principal series or of G intertwinning maps between the Schwartz 
space of the nilpotent cone and these degenerate principal series. 

2 . Conjecture 2 implies Conjecture 1 

This is a well known argument but we recall the proof. A group of type Ictd is 
a locally compact, totally discontinuous group which is countable at infinity. 
We consider smooth representations of such groups. If (tt, Ej^) is such a 
representation then (tt*, E*) is the smooth contragredient. Smooth induction 
is denoted by Ind and compact induction by ind. For any topological space T 
of type Ictd, S{T) is the space of functions locally constant, complex valued, 
defined on T and with compact support. 

Proposition 2.1 Let M be a tdlc group and N a closed subgroup, both uni- 
modular. Suppose that there exists an involutive anti-automorphism a of M 
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such that <j{N) = N and such that any distribution on M, biinvariant under 
N, is fixed by a. Then, for any irreducible admissible representation n of M 

dim (HomM(«nrfj^^(l),7r)) x dim (HomM(mrf*f (1), vr*)) < 1 

Let be the space of tt and i?* its smooth dual. Let A (resp. A') be 
a hnear map from S{M/N) into E,^ (resp. E*) commuting with M. For 
f,j2eS{M/N) put 

B{f,,f2)^{Af,,A'f2) 
Choose Haar measures on M and N. For 99 G S{M) define 



(p^{mN) — / (p(mn)dn 
Jn 



This is a surjective map from S{M) onto S{M/N). Then 

is a bihnear form on M x M left invariant under the action (mi, 7712) 1— > 
(mmi,mm2) of M onto MxM and right invariant under the action of 
N X N. View it as a distribution 5* on M x M and let us use integral 
notations 

B{fi,<f2)= / (pi{mi)ip2{m2)dS{mi,m2) 

J MxM 

Next we consider the homeomorphism of M x M onto itself; 

(mi, 777.2) I— > (m2 ^7«i, 7^2) 

The distribution 

(p ^ I (f o (^{mi,m2)dS{mi,m2) = / (p{m2^mi,m2)dS{mi,m2) 

JmxM JmxM 

is invariant under the action of M by left translations on the second variable 
and is biinvariant under the action of acting on the first variable by left and 
right multiplication. Hence there exists on M a distribution T, biinvariant 
under N and such that 



/ 

J MxM 



(p{m2^mi,m2)dS{mi,m2) = / </?(mi, m2)rfm2 dT(mi) 

J MxM 
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Suppose that </? is decomposed (p{mi,m2) = (pi{mi)(p2{m2). Let r : m 
a{m)~^; it is an automorphism of M leaving N stable. Define ^1{m) 
(pi{a{m)), (fi{m) = (pi{m~^) and (p1{m) = ipi{T{m)). Then 



ifi{mi)if2{rn2)dS{mi,m2) = / (pi{m2mi)ip2{m2)dm2dT{mi) 

MxM JMxM 

= (T, (pi * LP2) 



The last equality is due to the invariance of T under the anti automorphism 
(7. Working back we get 

{T,(pl*ipl)= / ipl{mi)ipl{m2)dS{mi,m2) 

JMxM 

As {(fil)^ = i^ly this implies that 

{Ah,A'f2)^{Af^,A'fl) 

Suppose that we can find a,n A' 0. Then A' is onto and Afi — ii and only 
if, for all /2 we have {Afi,A'f2) — 0. This means that A'f{ is orthogonal 
to the image of A. If A 7^ this means that A'fl — 0. Therefore if both 
A and A are non zeros then the kernel of A is the image under r of the 
kernel of A' . Because of Schur Lemma A and A' are, up to a constant factor 
determined by their respective kernels. So we are left with two possibilities 
for the dimensions dim {Rom.M{ind^{l),T:)) and dim {Rom.M{ind^{l),T^*)y. 
either they arc both equal to 1 or one of them is and the other arbitrary ( 
possibly infinite). 

Remark. — There is a variant for the non unimodular case; we will 
not need it. 

Corollary 2.1 Let M he a tdlc group and N a closed subgroup, both uni- 
modular. Suppose that there exists an involutive anti- automorphism a of M 
such that cr(iV) = N and such that any distribution on M, invariant under 
the adjoint action of N , is fixed by a. Then, for any irreducible admissible 
representation n of M and any irreducible admissible representation p of N 

dim (Homiv(7r|iv, p*)) x dim (Homiv((7r*)|iv, p)) < 1 
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Let M' = M X N and A^' the closed subgroup of M' image of the homomor- 
pism n 1-^ (n, n) of N into M. The map (m, n) i-^ mn~^ of M' onto M defines 
an homeomorphism of M' /N' onto M.The inverse map is m (m, 1)A^' On 
M'/N' left translations by N' correspond to the adjoint action of onto 
M. We have a bijection between the space of distributions T on M invariant 
under the adjoint action of N and the space of distributions S on M' which 
are biinvariant under N'. Explicitly 



Jn 

Suppose that T is invariant under a and consider the involutive anti- automorphism 
a' of M' given by a'{m, n) — {cr{m), cr{ri)). Then 



Jn 

Using the invariance under a and for the adjoint action of N we get 



Hence S is invariant under a'. Conversely if S is invariant under a' the same 
computation shows that T is invariant under a. Under the assumption of 
Corollary 2-1 we can now apply Proposition 2-1 and we obtain the inequality 



dim iB.omM'{ind^! {l),^ ® p)] x dim lHomM' {ind^! {l),7i* ^ p*)) <1 






(SJ) 



We know that Ind^',{l) is the smooth contragredient representation of indj^,'(l)hence 

Y{.omM'{ind%{l),n* ® p*) k, HomM'(7r ® p,Ind%{l)) 



Frobenius reciprocity tells us that 

HomM'(7r (g) p, Ind^',{l)) ^ Homjv'(7r (8) p)\n', 1) 



Clearly 



HomAr'(7r (g) p)iN', 1) ~ Homjv(p, (7r|jv)*) Homjv(7r|jv, p*) 
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Using again Probenius reciprocity we get 



HomAr(p, (ttijv)*) fti Rom m {iud ^ (p) , TV*) 

In the above computations we may replace p by p* and tt by tt*. Finally 

RouiM' {ind^'f {l),7r* (g) p*) ^ Hom7v(p, {t^in)*) 

^ Homjv(7r|jv,p*) 

HomM {ind^ (p) , tt* ) 
HomM'(mo?j|f/'(l),7r(8)p) HomAr(p*, ((7r*)|iv)*) 

Homiv((vr*)|Af,p) 
HomM (indjlf (p* ) , tt) 

Going back to the situation of section 1 we take M — GL{W) and N — 
GL{V). let E'tt be the spaces of the representation vr and let E* be the smooth 
dual (relative to the action of GL{W). Let Ep be the space of p and E* be 
the smooth dual for the action of GL{y). We know that the contragredient 
representation tt* in E* is equivalent to the representation g i— > n{^g^^) in 
E'tt- The same is true for p*. Therefore an element of Hom7v(7r|^, p*) may be 
described as a hnear map A from E„ into Ep such that, ior g e N 

Anig) = pCg-')A 

An element of HomAr((7r*)|jv, p) may be described as a linear map A' from 
E'tt into Ep such that, ior g e N 

A'TrCg-') = P{9)A' 
We have obtained the same set of hnear maps: 

Homiv((7r*)|Ar,p) « HomAr(7r|Ar, p*) 

We are left with 2 possibilities: either both spaces have dimension or they 
both have dimension 1 which is exactly what we want. 

Prom now on we forget Conjecture 1 and try to prove Conjecture 2 

3 . A partial linearization 

We keep the notations of section 1. Put G — GL{V). Let V* be the 
dual space of V. Choose a bijective linear map u from V to V* which is 
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self- adjoint. On G x V x V* we consider the involution a: 



The group G acts on G x V x V* by 



g{x,v,v*) ^ {gxg ^,gv,^g ^v*) 



We are going to show that Conjecture 2 is in turn implied by Conjecture 3 
Renicirk. — 1) The involution cr depends on the choice of u. However 
the action on invariant distributions does not so that the choice is irrelevant. 
If we need a precise u we start with a basis Ci, . . . , e„ of V and the dual basis 
el, . . . and define u by u{ei) = e*. 

Assume Conjecture 3. We shall apply it to GL{W) acting on GL(W) x 
W X W*. Let ei, . . . , e„ be a basis of V as above and choose e„+i e U,a, 
non zero vector. Call e^, . . . e*^^ the dual basis of W*. Define u, hence a, 
using this basis. Let Y be the set of all triples {g,w,w*) e GL{W) x W x 
W* such that {w*,w) = 1; it is a closed subset, stable under a and under 
GL(W). Therefore, by Conjecture 3, any G'L(VF)— invariant distribution on 
Y is fixed by a. Now let X be the closed subset of Y defined by w = e„+i 
and w* = e^^p Clearly GL{W)X = F and if ^ G X and ^ G GL{W) are 
such that g^ & X then g G GL{V) and gX = X and any g G GL{V) has this 
property. This means that we can use Probenius descent as described in the 
Appendix : there is a one to one correspondence between G'L(l^)— invariant 
distributions on Y and GL(y)— invariant distributions on X. If we identify X 
with GL[W) and if we choose an invariant measure on GL{W)/GL(y) then, 
to a GL(\^)— invariant distribution S on GL{W) corresponds the distribution 
T on y given by 



JGL{W)/GL{V) 

Now g H- > *g~^ u is an involutive automorphisme of GL{W) which defines 
an involutive homeomorphism of GL{W)/GL{y). Changing variables : 




If we replace S by (t{S) we get the distribution T': 





.+i)dg 
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which is equal to (T,a(/)) so that T' = a{T). We assumed that a{T) = T 
and the map S ^ T is one to one so (t{S) = S. 

We let G = GL{V) acts on its Lie algebra by the adjoint action and 
consider the action on q x V x V* . Clearly the involution a acts also on q. 

We are going to show that Conjecture 4 implies Conjecture 3. The basic 
tool is Harish-Chandra's descent method. The proof is by induction on the 
dimension n of V. So let us assume that Conjectures 3 and 4 are true for all 
local fields F non archimedean of characteristic and dim V < n. We take 
V of dimension n. 

4 . The case n = 1 

We take V — ¥ and also V* — F, the duality being the usual product. 
The group GL(y) is F*. It acts trivially on itself and on V x V* ^ ¥^ by 
{x,y) 1-^ {tx,t~^y). The involution a is {t,x,y) {t,y,x). We have to show 
that any distribution on F^, invariant under the action of F* is invariant by 
the map (x, y) i— > {y, x). Let f2 C F^ be the open subset xy ^ 0. The fibers of 
the map (x, y) ^ xy oi^ onto F* are the orbits of F*. Each of them is fixed 
under a. It follows from Bernstein's localization principle that a distribution 
on f), invariant under F* is invariant under a. If T is a distribution on F^, 
invariant under F* and such that o"(T) = — T its support must be contained 
in the closed subset Z : xy = 0. Then there exist two constants a and b such 



One possible extension of the right hand side to a distribution on Z is 



{T'J) = a if{x,0)-f{0,0)x{x))d*x + b ifiO,y)-f{0,0)x{y))d*y 



where x is the charactereistic function of the ring of integers in F. Then 
T — T' is a multiple of the Dirac measure at the origin hence is invariant 
under a and F*. On the other hand a trivial computation shows that T' is 
invariant by F* if and only if a = 6. On Z \ {(0, 0)} the distributions T and 
T' are equal so T' is skew with respect to a which means that a — —b. Thus 
we must have a = b = and , as there is no skew distributions supported by 
the origin we conclude that T — 0. In other words the space of F*— invariant 



that, on Z\ {(0,0)} 
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distributions on Z has dimension 2 and a basis consists of the Dirac measure 
at the origin and the distribution 

I (/(x,0)-/(0,0)x(a;))rf*a;+ / (/(O, y) - /(O, 0)x(2/)K2/ 

JF* JF* 

Invariance under F* forces symmetry under a\ 

5 . Harish- Chandra's descent 

Just for this section we work in a more general setup. Let F be a local 
field, of characteristic and non archimedean. Let g be a reductive Lie 
algebra, defined over F and G an algebraic group, defined over F and of Lie 
algebra g. 

For X G we denote by Z^{x) the centralizer of x in g and by Zg{x) its 
centralizer in G. 

The group G acts on g by the adjoint action; we write gx instead of 

Let a be a non central semi-simple element of g. Put 

m = Z^{a), M = Zcia) 

Let q be the image of ad(a); we have g = m © q. For x G m 

ad(a;)m C m, ad(a;)q C q 

Let m' be the set of all a; G m such that (ada;)|q is invertible. It is a Zariski 
open set in m. Choose a complete set . . . , 1)^ of representative of conjugacy 
classes of Cartan subalgebras of m (the group which acts is thus M). If G 
acts continuously on some topological space T, then a G— domain in T is 
a subset of T which is closed, open and invariant under G. The following 
Lemma is due to Harish- Chandra [4J, Corollary 2-3 

Lemma 5.1 Let u be a neighbourhood of a in m. There exists an M— domain 
Q in m such that 

a) a efl <Z Muj, 

b) fli)i C u for i = 1, . . . ,r, 

c) Q C va', 

d) for all compact sets Q of g there exists a compact set C in G such that 
xQQ 7^ implies x E C 



14 



Fix such an Q. Consider the map {g,x) ^ gx from G x Q into q. The 
differential of this map is 



{X,Y)^g{[X,x] + Y), Xeg,Yem 

It is onto if and only if m + Im(ada;) = g which means that x G m'. As we 
know that f2 C m' the map is everywhere submersive. In particular its image 
GQ is open. It follows from condition d) of Lemma 5-1 that GQ is also closed 
m, Corollary 2-4. 

Let E he a finite dimensional rational G— module (defined over F). The 
map {g, x, e) ^ {gx, ge) from G xQx E into gx E is everywhere submersive 
so that we may apply Harish-Chandra submersion principle. There exists 
a map / F/ of S{G x x E) onto «S(Ad(G')f2 x E) such that for any 
ip e 5(Ad(G)fi X E) 

/ f{g, X, e)(p{gxg~^, ge)dg dx de = / Ff{X, e)(p{X, e)dX de 

JGxnxE JAd{G)nxE 

Here dg is a Haar measure of G, dx the restriction to of a Haar measure 
of m, de a Haar measure of E and dX the restriction to Ad{G)Q of a Haar 
measure of g. By transposition we then get a one to one map T i— >■ $(T) 
of S'{Ad{G)n X E) into S'{G xVtx E). We let G acts on G x n x E 
by left translations on the first factor and on Ad(G)fi x ii^ by adjoint on 
the first factor and the given action on the second factor so that the map 
f Ff commutes with the action of G. It follows that if T is invariant 
then $(T) is also invariant and therefore may be written in a unique way as 
$(T) = dg0 e{T) with e{T) G S'{n X E). If we define 

Hf{x,e) = / f{g,x,e)dg 
Jg 

then {T,Ff) = {6{T), Hf) . The map 6 is one to one. Furthermore the 
group M acts on Q by the adjoint action and it acts on E. The distribution 
0{T) is semi-invariant. Indeed let ip ^ S{Q x E) and choose / such that 
ip = Hf. Fix m & M and define ipi{x, e) = %l){mxm~^ , me). If /i is given by 
fi{g, X, e) = f{g, mxm~^, me) then ipi = Hf^. To compute Fj^ we start from 

/ f{g,mxm~^x,me)ip{gxg~^,ge)dgdxde= / Ff-^{X, e)(f{X, e)dX de 

JGxnxE JAd{G)nxE 
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and on the left-hand side we change m into m ^xm and e into m which 
gives 

/ f{g, X, e)(p{gm~^xmg~^ , gm~^e)dg dx de |DetEm|~^ 
and now we change g into gm: 

I f{gm, X, e)(p{gxg~^, ge)dg dx de |DetBm|"^ 

JgxQxE 

Define f2{g,x,m) — f{gm,x,e). We have Ff^ — |Det£;m|F/j. Now 

{e{T),^P,) = {T,Ff,) = {T,Fj,)DetEm\-^ = (<I>(T), /2)|Detsm|-i 
= ($(T),/i)|DetEm|-i = (T, ^i)|DetEm|-i 

Proposition 5.1 The map is a one to one map from the space S{Gfl x 
E) ^ of G— invariant distributions on GVt x E into the space of distributions 
on fl X E having the above semi-invariance. 

Similar results are true for the group. Let a be a semi-simple element of 
G; we do not exclude the case a central. Let M be the centralizer of a in G 
and m its centralizer in q. The endomorphism Id — Ad{a) of g is semi-simple 
so that g = m © q where q is the image of Id — Ad{a). For any m E M one 
has Id — Adm(q C q and if m is sufficiently close to the neutral element of 
M then the restriction to q of Id — Adm is bijective. Choose a linearization 
G C GLn{¥) of G; then q C 0tAr(]F) and we have an exponential map defined 
on a subset of q. 

Lemma 5.2 There exists a neighborhood cu of in m with the following 

properties: 

a) uj is open, closed and invariant under the action of M 

b ) the exponential map is defined on lu and submersive at each point of cu 

c) for any X & cu the restriction to q of Id — Ad(ae^) is bijective. 

Indeed there exists in 5tAr(F) a GLjv(F)— invariant open neighborhood U ofO 
such that the exponential map is defined on U and is an analytic isomorphism 
of U onto Exp(C/). Choose an open neighborhood loq of in m such that 
Uq G mdU and such that condition c) of the Lemma is satisfied for X E ujq 
hence also for X e Ad(M)a;o. Then we take for cu an open, closed and 
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M— invariant neighborhood of such that uo C Ad(M)cJo ( see Lemma 5-1). 
Let Vt = Exp(c<j); it is an open invariant neighborhood of the identity in M 
( be careful that Q does not have the same meaning as before!). 
Let E he a finite dimensional rational G— module. 

Lemma 5.3 The map [g, m, e) i-^ {gmag~^, ge) of G x fix E into G x E is 
everywhere submersive. 

The differential is a map from q®vc\.®E into q®E. To prove the surjectivity 
we compute the partial differentials. First for (51,6), let (X, Z) e 5 ® £^ then 

gE-x^{tX){e + uZ) = g{e + tX + uZ ^ ) 

gE-x^{tX)maE-x^{-tX)g~'^ = ^mac/~^Exp (tAd(5r)Ad(m~^a~^)(Id - Ad{ma))X H ) 

Then the image of this partial differential is 

(Ad(^m-^a-^)Im(Id - Ad(ma))) ® E 

The image of Id — Ad(ma) contains q and q is fixed by Ad(m~^a~^) so the 
image of he differential contains the subspace Ad(5()(q) © E. 
Next we take y e m, then 

gme'K^{tY)ag~^ = gmag~^Exp{tAd{g)Y) 

and so the image of the differential contains Ad(g')m. Thus the differential 

is onto, as required. 

Now we apply Harish- Chandra submersion principle: there exists a sur- 
jective linear map / 1-^ F/ of S{G x Q x E) onto S{Ad{G){fla) x E) such 
that, for any ^ E S{Ad{G){na) x E) 

I f{g,m,e)(p{gmag~^,ge)dgdmde= / Ff(x,e)(p{x,e)dx de 

JcxflxE J Ad{G){na)xE 

On the left hand side dm is a Har measure on M. dg a Haar measure on G and 
de on E. On the right hand side dx is the restriction of dg to Ad(G')r2. By 
transposition we have a one to one linear map T 1-^ $(T) of 5'(Ad(G)(r2a) x 
E) into S'{G xnxE). The group G acts on Ad(G')(fia) x £; by the adjoint 
action on Ad(G') (Qa) and the given action on E and we let it act on GxQxE 
by left translations on the second component. As the map f ^ Ff commutes 
with these actions, if the distribution T is invariant so is the distribution 
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$(T). In that case there is a unique distribution 9{T) G <S'(Q x E) such that 
$(r) = (g) ^(T). If we define 



Hf{m,e) = / f{g,m,e)dg 
Jg 



the^n {T,Ff) = {eiT),Hf). 

The group M acts on Q by adjoint action and so it acts on Q x E. If 
uE M then 

Hf{umu'^,ue)= I f{g,umu'^,ue)dg 
Jg 

Define f'{gi m, e) = f{g, umu~^, ue) and /"(g', m, e) = /(sfw, m, e). Then 

IgxQxe f(9: umu-^, ue)ip{gmag-^, ge)dgdmde 
= \T>eiEu\~^ JgxQxe f(9U: m, e)(p{gmag~^, ge)dgdmde 

which means that Fj/ = |Det£n|^^Fj//. Therefore {T,Ff/) = \DetEu\~^{T, Ffn) 
and then ($(T),/') = |DetEM|-i($(T), /"). However $(T) = dc/ ® ^(T) im- 
plies that ($(T),/") = mT)J) so that {0{T),Hf,) = |DetBii|-i(^(T), /): 
the distribution ^(T) is semi-invariant under M; in integral notations: 

/ 'ilj{umu~^,ue) d9{T){m,e) — \DetEu\~^ / ip{m,e) d9{T){m,e) 

jQ.y.E JQ.y.E 



Proposition 5.2 T/ie ma|? 9 is a one to one linear map from the space 
S'{Kd{G){VLa) X E)^ of G— invariant distributions on Ad{G){VLa) x E into 
the space of distributions on x E with the above semi-invariance property. 

Now we go back to our setup: G = GL{V) and E = V (B V*. In this case 
Betsg = 1 Let a be a semi-simple element of EndF(V^). We want to describe 
the centralizer m of a in g and the centralizer M of a in the linear group G. 
Let P be its minimal polynomial; all its roots are simple. Let P = Pi. . .Pj. 
be the decomposition of P into irreducible factors, over F. Then the Pj are 
two by two relatively prime. If = Ker Pj (a), then V = ©VJ and V* = (BV*. 
An element x of G which commutes with a is given by a familly {xi, . . . ,Xr} 
where each linear map from to V^, commuting with the restriction 

of a to Vi. Now ¥[T] acts on Vi, by specializing T to a|y. and Pj acts trivially 
so that, if Fj — F[r]/(Pj), then Vi becomes a vector space over Fj. The 
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F— linear map Xi commutes with a if and only if it is Fj— linear. 

Fix i. Let £ be a non zero F— linear form on Fj. If Vi G Vi and v[ G V* then 
A I— > (Auj, v'^ is an F— linear form on Fj, hence there exists a unique element 
S{vi,v[) of Fj such that {\vi,v[) = £ (AS'(vi, v-)). One checks trivially that 
S is Fj— linear with respect to each variable and defines a non degenerate 
duality, over Fj between Vi and V* . Here Fj acts on V* by transposition, 
relative to the F— duality (., .) ,of the action on Vj. Finally if Xi G Endp^Vj its 
transpose, relative to the duality .) is the same as its transpose relative 
to the duality (.,.). 

Thus M is a product of linear groups and the situation (M, V, V*) is a 
composite case, each component being a linear case (over various extensions 
of F). 

Recall that the involution on g x V x V* or on G x V" x V*is not unique 
(although the action on the space of invariant distributions is) .We choose a 
basis of V that is to say an isomorphism (/? of F'* onto V onto V, identify F" 
with its dual and then put u (p o (p. The involution is then given by 

{x,v,v*) ^ {u^^ ^Xu,u'^{v*),u{v)) 

We assume that ip is compatible with the decomposition V = ©Vj in the 
sense that the image by p of the canonical basis of F" is the union of basis 
of the Vi- We are going to define aa & G such that aa u~^au — a. This 
can be done independantly for each Vj. 

Fix i. Let s be the dimension of Vj over Fj and choose an isomorphism $ 
of F| onto Vj, then identify Fj with its dual and put Ui =* $$. 

Let U be the direct sum of the Ui. Note that Ua = *a U^^. Call s the 
involution on m x V^ x V^* or on M x V^ x V^* product of the involutions built 
with the Ui on each factor. 

Let us look at the situation on the group. Let ip G S(yt x y x V^*); choose 
/ G S{G xVtxV xV*) such that V = Hf. Let Vi be defined by 

V^i(m, V, V*) = ipiU-^ *m [/, U-\v*), U{v)) 

This make sense because any M— orbit (resp. G— orbit) in M (resp. in G) is 
stable by the involution m i— >• U~^ *m U (resp .g h- > *g u) 
Then = Hf^ with 

h{g, m, V, V*) = f{g, U'' *m U, U-\v*), U{v)) 
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We want to compute Ff^ . We have 

/gx^xV-xV* /(^' ^'^ ^' ^~^(^*)' ^ {v))'^{9mag-\ gv,^ g~^v*)dg dm dv dv* 
= loxaxYxV* fid, V, v*)(p{gU-^ *m f/ a^f-^ gu'^v*,^ g-^Uv)dg dm dv dv* 
= !gxQxVxv* /(^"^ ^' ^' ^' v*)(p{u-^\gmag-^)u, u'^ ^g~^v\ guv)dg dx dv dv* 

The last cquahty corresponding to the involutive change of variables g i— > 
^g~^ U . Then define /2 by 

f2{g, m, V, V*) = f{u~^ ^g-^ U, m, v, v*). 

The above integral is equal to 

lAd(G){na)xVxv* ^12(9, V, v*)ip{u-^ ^gu, u-l{v*),u{v))dg dv dv* 
= lAd(GKna)xVxv* ^hi^'^ '9 u, u-\v*) , u{v))ip{g , V, v*)dg dv dv* 

Finally 

Ff,{g,v,v*) = Ff,{u-^'gu,u-\v*),u{v)) 

If T is an invariant distribution on Ad{G){fla) x V x V*, skew symmetric 
with respect to the involution a then 

{T,Ff,)^-{T,Ff,). 

Therefore 

{e{T),^,) = -{e{T),Hf,). 

However Hf^ = Hf = ip so the distribution 9{T) is skew symmetric with 
respect to the involution s. 

The exponential map is an isomorphism of cu onto Q and commutes with 
the adjoint action of M. We may view 9{T) as an invariant distribution on 
uj X V X V* but uj is closed in m so 0{T) extends to an invariant distribution 
on m X V X V*. If we assume Conjecture 4 then 9{T) must be symmetric, 
hence and T = 0. 

Now let T be an invariant distribution on G x y x y* which is skew 
with respect to a and let {g,v,v*) be any element of G x y x V* . Take 
for a the semi-simple part of g. Then , always assuming Conjecture 4, the 
distribution T will be on Ad{G){Qa) x V x V*. However a belongs to the 
closure of the orbit of g so, for some g' E G we have {g'gg'~^, g'v,^ g'~^v*) G 
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Ad{G){Qa) X V X V* and we conclude that T = in a neighborhood of 

{Qj V, V*) and finally T = 0. 

Conclusion. — Conjecture 4 implies Conjecture 3 

We now deal, in exactly the same way, with the Lie algebra case. We 
assume that a is not central so that, by the induction hypothesis. Conjecture 
4 is vahd for each i and hence for the composite case m ®V ®V*. We use 
the notations of Proposition 5-2. 

Let eS{nxVxV*y, choose / e S{G x^xYxV*) such that = Hf. 
Let ipi be defined by 

ijjiix, V, V*) = ipiU-^ *x U, U-\v*), U(v)) 

This make sense because any M— orbit (resp. G— orbit) in m (resp. in q) is 
stable by the involution x i— > *x U (resp .X i— > *Xu) Then tpi — Hf^ 
with 

h{g, X, V, V*) = f{g, 'x U, U-\v*), U{v)) 
We want to compute Ff^ . We have 

/gx^xv-x V /(^' *^ ^' U-\v*), U {v))ip{gxg-\ gv,^ g-^v*)dg dx dv dv* 
^ IcxUxYxv* f(9, X, V, v*)ip{gU-^ ^xUg'^, gu'^v* g-^Uv)dg dx dv dv* 
= IcxUxYxv /(""^ ^9"^ v*)ip{u'^ \gxg'^)u, u'^ ^g~^v*, guv)dg dx dv dv* 

The last equality corresponding to the involutive change of variables g i— > 
*g~^ U. Then define /2 by 

f2(g, X, V, V*) = f(u~^ *g~^ U, x, v, v*). 

The above integral is equal to 

lAd{G)nxVxV* ^hi^^ *^ u-l{v*),u{v))dX dv dv* 

= lAd{G)nxVxV* *^ u-\v*),u{v))ip{X, v, v*)dX dv dv* 

Finally 

Ff, (X, V, V*) = Ff, (u-^ 'X u, u-\v*),uiv)) 

If T is an invariant distribution on Ad{G)fl x V xV*, skew symmetric with 
respect to the involution a then 

{T,Ff,)^-{T,Ff,). 
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Therefore 

However Hf^ = Hf = i/j so the distribution 6{T) is skew symmetric with 
respect to the involution s, but Q x V x V* is open and close so that 0{T) 
may be considered as a distribution on m x V x V* and, by induction, is 
symmetric so it must be and T = 0. 

Now Ad{G)Q xV xV* is open in g x y x y* so if T is now an invariant 
distribution on q x V x V* , skew symmetric with respect to a we conclude 
that its restriction to Ad{G)Q x V x V* is 0. If {X,v,v*) belongs to the 
support of such a distribution and if the semi-simple part a of X is not 
central then, with the above notations, T is on Ad(G)f2 x V x V*. We 
know that a belongs to the closure of the orbit of X, hence there exists g E G 
such that {gXg-^,gv,^g-^v*) E Ad(G)fi xV xV*. It follows that the orbit 
of {X, V, V*) is contained in Ad{G)0, xV xV* and so T is in a neighborhood 
of {X,v,v*). Thus: 

Conclusion. — If T is a skew symmetric invariant distribution on 
gxVxV* then its support is contained in ^ x J\f x V x V* where 3 
is the center of q and jV the cone of nilpotent elements in [g, g\ . 

6 . Regular orbits 

For V e V,v* e V* and X e put 

q,{X,v,v*) ^ {v*,X'v) ^ CX'v*,v) 

and 

n-l 

Det(rid - X) ^T"" - Dj{X)T^ 



The Qi and Di are invariant polynomial functions. Let 

A{X,v,v*) = (a,,,), = CX^-'v*,X'-'v), i,j = 1, . . .n 

an n X n matrix and put 

D{X,v,v*) = Det{A{X,v,v*)) 

A triplet {X, v, v*) is regular if {v, Xv, . . . X'^~^v} is a basis of V and 
{v*, ^Xv\ . . . , ^X^-i-u*} is a basis of V* . 
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Proposition 6.1 The triplet {X, v, v*) is regular if and only ifD{X, v, v*) ^ 
0. The set of regular elements is a non empty Zariski open subset. 

On V ®V* the bilinear form 

((v, V*), (w, w*)) {v*, w) + (w*, v) 
is symmetric and non degenerate. If {X,v,v*) is regular then 

is a basis of V®V* and, relative to this basis, the matrix of the above bilinear 
form is 

A 
A 

and is non singular which means that A is non singular. Conversely, any non 
trivial hnear relation among the X'^v, i — . . .n — 1 (resp the *'X^v*, j — 
0,...n — 1) gives a non trivial linear relation among the rows (resp the 
columns) of A so that if the triplet is not regular then D{X,v,v*) = 0. 

To prove the second assertion we simply have to exhibit a regular clement. 
For example choose a basis ei, . . . e„ of ^ and let e];, . . . , e* be the dual basis 
of V*. Then define X by 



X{ei) — Cj+i, i = 1, . . . n — 1, X{en) = and v = ei, v* — e, 

Then 



n 



'Xe*^eU, i^2,...,n, 'Xel^O 
it follows that {X,v,v*) is regular. 

Proposition 6.2 Two regular elements are conjugate under G if and only 
if they give the same values to the invariants qj and Dj. 

The necessity is clear. Conversely let {X, v, v*) and (F, w, w*) be two regular 
elements such that 

qj{X,v,v*) ^ qj{Y,w,w*), Dj{X,v,v*) ^ Dj{Y,w,w*) j = 

In particular 

A{X,v,v*) ^A{Y,w,w*) 



23 



Let g be the linear map from V to V defined by g{X^v) = Y^w for p — 
0, . . . n — 1. We claim that gXg~^ — Y. It is enough to check that 

g-^YgXPy = X^+^v, p = 0, . . . n - 1 

Now, if p < n — 2 

g-^YgXPy = g-^Y^+^w = XP+\ 

For p — n — 1 

n—l n—l 

g-^YgX''-^ = g-^Y^'w = ^"^ J]] Dj{Y)Y^w = J]] Dj(X)X^t; = X^'v 



Also 

(V^'y*,^M = {v*,g~^Y^w) = {v*,X^v) = qj{X,v,v*) 

and 

{w*,Y^w) = qj{Y,w,w*) 

Hence, for all p we get 

{^g-^v*,Y^w) = {w*,Y^w) 

which implies that ^g^^v* = w*. The two triplets arc conjugate by g. 

We define the regular orbits as the orbits of regular elements. Each 
such orbit is defined by the values of the qj and the Dj. These values must 
be such that D, which is a polynomial in the qj should not take the value 
and also the relations between the qj and Dj must be satisfied by the chosen 
values. Each such orbit is (Zariski) closed (the invariants arc constant on 
the closure of any orbit !). If {X,v,v*) is regular and if 5^ G G is such 
that g{X,v,v*) = {X,v,v*) which means in particular that gv — v and 
gXg~^ = X, then gX^v = {gXg~^Ygv = X^v for all p. By definition of 
regular {v, Xv, . . . , } is a basis of V so we conclude that g — Id: the 

isotropy subgroup of a regular element is trivial. In fact 

Theorem 6.1 An orbit is regular if and only if it is closed and if the cen- 
tralizer in G of an element of the orbit is trivial. . 
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Let {X,v,v*) be a non regular triplet. Suppose for example that the sub- 
space E of V generated by v, Xv, . . . , X"^-*^?; is a proper subspace. Choose 
a subspace F such that V = E (B F. With respect to this decomposition we 
can write X as 

'A B 
C 



X 



For te F* define 



Then v is fixed by at and 



Ids 



atXa^ ^ 



A tB 
C 



so that 



A 



lim a^Xa^ ^ 



Furthermore we have = i?* © F*; decompose = ujl' + Then 
so that 

lim ttf V — 

Note that the limit point (when t goes to ) of {X, v, v*) is fixed by at- Thus 
either the orbit is not closed or, if it is closed the centralizer of one of its 
point is not trivial. 

We can also give a characterization of the closed orbits, as before let 
E be the subspace generated by v, Xv, X'^v, . . . and let E' be the subspace 
generated by v*, ^Xv*, ^X^v*, .... Choose a subspace F of V such that V = 
E ® F. Then, relative to this decomposition, we may write 



X 



A B 
C 



If we view C as a linear map from V/ E into itself then it does not depend 
upon the choice of F. 

Theorem 6.2 The orbit of {X, v, v*) is closed if and only if V* ^ E' ® E^ 
and C is semi-simple. 
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The condition V* = E' ® is equivalent to the condition V = E ® E'-^ 
and means that E' (resp. E) may be identified with the dual space E* (resp 
E *) of E (resp E'). Suppose that this condition is not satisfied and that, for 
example dim£" ^ dimE Define at as before. Then 

lim{X,v,v*)^{Y,v,e*) 

where e* is obtained as follows: we identified V* with E* © F* and write 
V* — e* + f* and 

(A 

\o c 

The subspace E'{Y, e*) generated by e*, ^Ye*, ... is contained in E* so that 

dim E'{Y, e*) < dim E* = dim < dim E' 

If {X,v,v*) is conjugate to {Y,v,e*) we must have dim£"(y, e*) = dim£" so 
that E'{Y,e*) = E*. But the subspace E(Y,v) generated by v,Yv, ... is 
so that (y, v,e*) satisifes the first condition of the theorem, hence can not be 
conjugate to {X, v, v*). 

Thus, looking for closed orbits, we may assume that V* = E' ® E-^ and 
V = E®E'^. So we choose F = E'^ and identify F* with E^ and E* with 
E'. Now 

M 



'X 



However E' — E* is stable under *X so that B — 0. 

Suppose first that the orbit of (X, v, v*) is closed. Let D be some element 
in the closure of the orbit GL(F)C. There exists a sequence of elements 
of GL(F) such that UnCu-^ D. Put 

Ids 
u„ 



Then 

QnV = V, *g~^v* = V*, QnXg'^ ^ 
As the orbit is closed there exists g & G such that 

t —1 * * — 1 

gv = v, g V =v , gXg = 



A 
D 



A 
D 
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Now X'^v = A'^v and g(Xiv) = (gXg''^yg{v) = A'^v so that g is the identity 
on E and, by the same argument, *g~^ is the identity on E*. Finally g may 
be written as 




and D — SCS"^ which shows that D belongs to the orbit of C. Hence this 
orbit is closed which is equivalent to the semi-simplicity of C. 

Conversely assume that C is semi-simple. Let ((?„) be a sequence of 
elements of G such that gn{X,v,v*) has a limit (Y,u,u*). Consider the sub- 
space E{Y, u) generated by m, Yu, Y^u, . . .. For any q the sequence {gn{X'^v)) 
converges to Y'^u. Hence any linear relation between the vectors X'^v re- 
mains valid for the Y'^u. In particular if E is of dimension p then E{Y, u) 
is of dimension at most p, the set {v,Xv. . . . ,X^~^v} is a basis of E and 
the set {u,Yu, . . . ,Y^~^u} is a set of generators of E{Y,u). The matrix 
{{^X'^~-^v* , X^~-^v)) , < i,j < p — 1 is invariant under G and, as E' f» E* 
is non singular. Hence the matrix {{*Y'^~^u* ,Y^~^u)) , < i,j < p — 1 is 
also non singular. This implies that the vectors u, Yu, . . . , Y^'^u are linearly 
independant. Similarly we prove that E'(^Y. u) the subspacc of V* generated 
by u*, ^Yu*, . . . has dimension p and admits {u*, ^Yu*, . . . , ^Y^^^u*} as basis. 
Also the pairing between E{Y,u) and E'{^Y,u*) is non singular. 

Choose g eG such that g{F) = E'{%u*)^ and g{X"v) = Y^u, Q<q< 
p-1. Note that e E'{%u*).We have 

Cg-''X''v*,gX'-v) = {'X'^v^X^'v) = {'Y^u^Y'u) 

and 

{^g-^*Xiv*,gX'v) = {^g-^^Xiv*,Y'u) 

This being true for < r < p — 1 implies that ^g~^^X'^v* — *Y'^u* . 

If we replace the sequence ((?„) by the sequence g'^gn we see that we can 
assume that, for all e e £^ and e* e E* one has 

limg'„(e) = e, lim^n^e* = e* 

In particular X^'v = Y'u for all r. Next consider the decomposition V = 
E e E*^ and put 

_ f Oin Pn\ -1 _ f ttn bn\ 
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We get 

limQ;„ = Idfi, lim7„ = 

Also 

From g^g:^^ = g^^gn = ld we get 

Oinbn + Pndn = 
771^*71 + Sndn = 1 

For n large enough q;„ is invertible so that we may write 
and 

which shows that d^is invertible with inverse — —jnCXn^Pn + ^n- 
From limg(„Xgf~^ = F we get 

anAbn + PnCdn ^ 

7„A6„ + 5„Cd„ ^ D 

Put 

/5„Cd„ = -Q;„A6n + En, lira En = 

Then 

D = lim(7„yl6„ + 6nCdn) = hm(7„A6„ + d;'^^Cdn + ^nOin^f^nCdn) 

= lim (7„A6„ + d^-^^Cdn + 0'nttn^(-"n^&n + ^n)) 

= lim((i^^C(i„ + 7„q;„ V„) 
— \im{d^^Cdn) 

As C is semi-simple its orbit under GL(£'*^) is closed. Hence, for some 
u e GL(£'*^) we have D = uCwK Let 



9 

We get 



Ids 
u 



g{X,v,v*) ^ {Y,u,u*) 



28 



The orbit of {X,v,v*) is closed. 

Let us go back to the regular orbits. The invariant polynomials and Dj 
are also invariant under the involution a. Then Proposition 6-2 implies that 
the regular orbits are also stable by a. For the p-adic topology each of them 
is homeomorphic to G and carries an invariant measure which is symmetric 
with respect to cr. It follows from a classical result of Gelfand and Kazdhan 
that a distribution on the regular set, invariant under G is symmetric with 
respect to a. Therefore an invariant distribution on gxV xV* which is skew 
symmetric with respect to a is supported in the complement of the regular 
set. 

We want to prove similar results for G acting on G xV xV*. 
For V e V,v* e V* and x eG put 

qi{x,v,v*) = {v*,x^v) = (*a;*w*,f) 

and 

n-l 

Det(rid - x) = - ^ Dj(x)T^ 



The Qi and are invariant polynomial functions, let 

A{x,v,v*) = {ttij), ttij = {^x^'''^v*,x^~'^v), i,j ^l,...n 
an n X n matrix and put 

D{x, V, V*) = Det{A{x, v, v*) 

A triplet {x,v,v*) is regular if {v,xv, . . .x"'~^v} is a basis of V and 

{v*, ^xv*, .... ^x"-^^v*} is a basis of V*. 

Proposition 6.3 The triplet (x.v.v*) is regular if and only if D{x,v,v*) ^ 
0. The set of regular elements is a non empty Zariski open subset. 

The proof is the same as the proof of Proposition 6-1 except that to exhibit a 
regular element we replace the principal nilpotent element X by the unipotent 
element x = 1 + X. 

Proposition 6.4 Two regular elements are conjugate under G if and only 
if they give the same values to the invariants qj and Dj . 
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Exactly the same proof as Proposition 6-2. In fact if we use the inclusion 
G C it is even a particular case. 

We define the regular orbits as the orbits of regular elements. Each such 
orbit is defined by the values of the qj and the Dj. These values must be such 
that D, which is a polynomial in the qj should not take the value and also 
the relations between the qj and Dj must be satisfied by the chosen values; 
also as we want x to be invertiblc, wc require that Dq ^ 0. Each such orbit 
is (Zariski) closed (the invariants arc constant on the closure of any orbit !). 
If {x,v,v*) is regular and if e G is such that g{x,v,v*) = {X,v,v*) which 
means in particular that gv — v and gxg~^ = x, then gx^v = {gxg~^Ygv — 
x^v for all p. By definition of regular {v, Xv, . . . , X'^'^v} is a basis of V 
so we conclude that g — Id: the isotropy subgroup of a regular element is 
trivial. In fact 

Theorem 6.3 An orbit is regular if and only if it is closed and if the cen- 

tralizer in G of an element of the orbit is trivial. . 

Same proof as for Theorem 6-1. If we use the fact that G C g it is a particular 
case because the determinant being constant on the closure of an orbit, the 
closure in g is the same as the closure in G. 

Finally we can also give a characterization of the closed orbits. Let E be 
the subspace generated by v, xv, x'^v, . . . and let E' be the subspace generated 

by V*, *xv*, ^x'^v*, Choose a subspace F oiV such that V = E®F. Then, 

relative to this decomposition, we may write 



If we view C as a linear map from V/ E into itself then it does not depend 
upon the choice of F. 

Theorem 6.4 The orbit of (a;,t',f*) is closed if and only ifV* = E' (B E-^ 
and G is semi-simple. 

We already remarked that the closure in g is equal to the closure in G so it 
is a particular case of Theorem 6-2. 

7 . A Frobenius type descent. 

With the same notations as in the preceeding section we want to show 
that an invariant distribution on g x V x V* , skew with respect to a must 




30 



be supported by the set of all {X, v, v*) such that {v*, X'^v) — for all i. 
For 1 <r <n define 

r— 1 r— 1 

Er{X, v) = Y^ ¥Xh, El{X, t;*) = ^ F 'Xh* 

j=0 j=0 

For 1 < r < n let S(r) be the set of all (x.v.v*) such that Ej.{X,v) and 
El.{X,v*) are both of dimension r and that (., .) defines a non degenerate 
duality between these two spaces. As before put 

qj = {v*, X^v), a,,, = CX^-\*, X'-\) = 

and let Ar{X^ v, v*) be the r x r matrix with coefficients Ojj for 1 < i, j < r. 
Then (X, v,v*) G S(r) if and only if the matrix Ar is non singular. It follows 
that S(r) is open; also is the subset of regular elements. 
Lemma 7.1 Each S(r) is an open subset. The complement of the union 
of all E(r) is the set of all {X,v,v*) such that E'^{X,v*) is contained in 

Indeed {X,v,v*) does not belong to E(l) if and only if qq — 0, does not 
belong to S(l) U S(2) if and only ii = qi — and so on. It does not 
belong to S(l) U ■ ■ ■ U if and only if qo = qi = ■ ■ ■ = = 0. This last 
condition implies that all qr = because X" is a linear combination of the 
X^' for j = 0, . . . , n - 1 

Fix r < n. Let Ehc a subspacc of V of dimension r and choose a subspace 
F such that V = F®E. identify V* with F*®E* so that, in particular E^ = 
F* and F = E*^. Let {X,v,v*) e E(r). Then V = E{X,v) ® E'{X,v*)^. 
Hence there exists g e G such that gE{X, v) = E and gE'{X, v*)^ — F which 
implies that ^g-^E'{X,v*) = E*. Let S(r) be the set of all {X,v,v*) e E(r) 
such that E{X, v) = E and E'{X, v*) = it is a closed subset and we just 
saw that GS(r) = S(r). Furthermore let {X,v,v*) G H(r) and suppose that 
for some g E G, we have g{X, v, v*) G S(r). put g{X, v, v*) = {X', v', v *) We 
know that {v, Xv, . . . , X'^~^v} and {v', X'v', . . . , X''^~^v'} are two basis of E 
and that g{X^v) — X'^v'. It follows that g{E) — E. The same argument 
shows that ^g{E*) = E*. If we use the decompositions V — F ® E and 
V* — F* Q) E* to write g and *^g as two by two matrices then 

-Co :) 
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with gp e GL(F) and qe G GL{E) so that gr.ir) = S(r). and we are in a 
situation to use the resuhs of the Appendix. We call H the stabilizer of S(r) 
in G. In matrix form, as above, it is the subgroup of diagonal matrices. 
We have to look in more details at S(r). Let {X,v,v*) e S(r) and put 

X = ( ^^'^ ^^'0 
We have xi^2X^v — for j — 0,1, ... ,r — 2. Let w* e E* be defined by 

r-l 


3^1,26 = {w*, e)u, e & E 
define w & E by 

r-l 


and 
to get 

*a;2,ie* = (e*,'u;)ii*, e* e E* 

Note that (xi^i, m, m*) e Qf x F x F* and (a;2,2, 'f, v*) e QeX E x E*. and is a 
regular element. Conversely if we start with two such elements then we can 
recover {X,v,v*) e S(r) by the above formulas. Thus if we denote by {qe x 
E X £'*)reg the open set of regular elements then we have an homeomorphism 

S(r) ^(qfxFx F*) X ((fls xEx E*),,^^ 

Next let us look at the action of H; take 

^=[0 He) 



and put 
Then 

In a similar way. we 
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Then h acts by 

u H- >• hpu^ u* ^ *hp^u*, 
V ^ hEV, V* I— ^h]^^v*, 

The action of H on 5(r) is the product of the action on the two components. 

Proposition 7.1 {X,v,v*) is regular if and only if {xi^i,u,u*) is regular. 

Indeed v, Xv, . . . , X^~^v is a basis of E and ,by a trivial induction, there 
exists constants Ajj such that 

= x'l^^u + Xs,ix'l'^i^u + . . . + Xs,sU (mod E) 

so that V, Xv, . . . , X^~^v is a basis of V if and only if u, xi^iu, .... x'^^'^'^^u is 
a basis of F. The same argument is valid for v* and u*, hence the result. 

To define the involution a relative to V we start from a basis of E and 
a basis of F. Then we have two maps ue and up from E and F to their 
respective dual space, sending the basis to the dual basis, we define u — 
{uf,ue)- Recall that 

a{X,v,v*) = {u-^^Xu, u-\v*), u{v)) 

Then E(r) is stable. We use Proposition 1 of the Appendix: for any distri- 
bution T on S(r), invariant under G, there exists a unique distribution S on 
S(r), invariant under H and such that, for / G iS(S(r)) 

(T,/)= / {S,f(gx)dM 
Jg/h 

In this case /i is an invariant measure on G/H. Now S(r) is homeomorphic 
to X F X F*^ X ^(5bx£'x -E'*)reg^ . The involution a fixes S(r) and its 
restriction is the product of the partial involutions up and ue- 

Proposition 7.2 Let {X,v,v*) e S(r). The orbit G{X,v,v*) is fixed by a if 
and only if the orbit GL(F)(xi,i, u, u*) is fixed by ap- If any Gp— invariant 
distribution on Qp x F x F* is invariant under up then then any G— invariant 
distribution on E(r) is invariant by a . 
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The first assertion follows from the above remarks and the fact that any 
regular orbit is fixed by the involution.. For the second we have to check 
that the map S* ^— T given by the Appendix is compatible with a. Recall 
that a o g = g' o a with g' = o *g~^ o u where g & G and, as before 



As a is an involution so is the map g ^ g' , hence the Haar measure of G is 
invariant under this map and the same is true for the stable subgroup H and 
so also for the quotient measure onG/H. If we replace S by (j{S) we obtain 



{S,f{ga{x))dii{g)^ {SJ{ag'x))dfx{g)^ {S, f{agx))dii{g) 



so that T is replaced by cr(T). 

As we proceed by induction on n we conclude that any invariant distri- 
bution on QxV xV*, skew with respect to a, is on E(r). This is true for 
any r ( the case r = n is the case of regular orbits). 

Let S C [g, g] X 1/ X \/* be the set of all (X, v, v*) such that X e Af, the 
nilpotent cone and that, for all i, {v*,X'^v) = 0. Recall that 3 is the center 
of g. 

Lemma 7.2 // any invariant distribution on [g, q] x V x V* is symmetric 
with respect to a then any invariant distribution on gxV xV* is symmetric 
with respect to a. 

This is clear as G and a act trivially on 3. 

Conclusion. — The support of an invariant distribution on qxV x 
V*, skew with respect to a is contained in E 

8 Invariant distributions with singular support 

Our first goal is to find extra conditions satisfied by these distributions. 

8.1 Some recollections 

Let E he a vector space of dimension n over F. Let Q be a non degenerate 
quadratic form on E and let 







B{X, Y) = Q{X + Y)- Q{X) - Q{Y) 
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We define the Fourier transform by 



/ f{X)r{B{X,Y))dX 
Je 



where r is a non trivial additive character of F and dX the Haar measure on 
E such that the inversion formula is 

f{X)= f f{Y)T(-B(X,Y))dY 
Je 

We denote by {a\b) he Hilbert symbol of F. The metaplectic group Mp2(F) 
is the twofold covering of SL2(F) defined as follows. If 



then we let j{g) = c if c 7^ and j{g) — d ii c — 0. For g, g' e SL2(F) and 
g" — gg' put 

e{9,g')^{j{9)j{g")\j{9'mg") 

Then an element of the metaplectic group is written as {g, e) with g e SL2(F) 
and e — ±1 and the group law is 

{9,£){9',£') = (9 9, ££'£{9, 9')) 

To the quadratic form Q ( and the fixed character r) is attached an eight 
root of unity j{Q) and there exists a (unique) representation ttq of Mp2(F) 
into S{E) such that 



1 u 
1 

t 

t-i 

1 
-1 



. e 



f{X) = e^T{uQ{X))f{X) 



litQ) 

f{X) = e-i{Q)f{X) 



t\''''f{tX) 



Let Fo be the cone of all X e E such that Q{X) = Q. 
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Lemma 8.1 Let T G S'{E) be such that both T and T have support con- 
tained in Tq. Then T is homogeneous: 



{T,f{tX))^\t\ 



-n/2 



(TJ) 



In particular if n is odd then T = 0. 
The condition on the support of T may be written as 

1 u 



1 



The condition on the support of T may be written as 

1 u 



1 



f)=e-{TJ) 



or 



(r,7(g)-VQ 



1 
-1 



,1 



1 u 
1 



f)^e^{T,j{Q)-' 



1 
-1 



or 



1 



1 



-1 OJ ' 
Computing the product we get 

with V = —u. 
But 



1 u 
1 









[C 











1 

-1 y ' 



1 -t 
1 



1 

t-' l''" 



t 


1 -t 

1 y ' 



1 1 
1 r 



,1 



1 \ 

-1 ir 
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Hence 



t 
t 



-1 



and we obtain the homogeneity of T. 

If T 7^ then 'y{tQ)/'y{Q) must be a multiphcative character which is 
true if and only if n is even. 

Note also that 



1 
-1 



1 



which implies 



or, explicitly 



1 
-1 



1 
1 



,1 



1 -1 

1 



-1 
1 



,1 



f)=e'\TJ) 



l{Q)T^T 

Finally, if n is even, let D be the discriminant of Q] then 



7(Q) 



8.2 Partial Fourier transform on [g, g] 

Now we go back to the GL— case. Let Qi = [g, g] be the derived algebra.The 
center will play no role hence, in this section, we systematically work with 
01. Let B be the Killing form of fli and consider the quadratic form R{Xi) — 
B{Xi^Xi)/2. We choose, once for all, a non trivial additive character r of 
F. In this section Fourier transform means partial Fourier transform with 
respect to gi and is denoted by / i-^ JF/: 



v*)t{B{X, Y))dX 



As B is invariant under G, this transformation commutes with the action of 
G. It also commutes with the involution a. Indeed if (cj) is a basis of V and 
(e*) the dual basis of V* and if u : V ^ V* is given by e* = u{ei) then we 
may normalize a by 



a{Xi,v,v*) = {u-^^Xiu,u-\v*),u{v)) 
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Hence 



J^{foa){Y,v,v*)= / f{u-''Xu,u-\v*),u{v))T{B{X,Y))dX 
J 01 

The measure dX is invariant by the involution X i— >■ ^Xu. For some 
c„ > we have B{X, Y) = c„Tr(Xr) so 

B{u-^^Xu,Y) = CnTr{u-^^XuY) = CnTrCXnYu'^) = CnTr{u-^^uX) 

which means that 

B{u-^^Xu,Y) = B{X,u-^*Yu) 

and imphes that 

^(/ o a){Y, V, V*) = Vk, «(^;)) 

It foUows that the Fourier transform of a distribution which is invariant under 
G and skew relative to a is again invariant and skew. Suppose that T is such 
a distribution. We know that the support of T is contained in the singular 
set and the same is true for JFT. Let a G S{V © V*); for G 5(gi) put 
Ta{^) = T{a ® (f ). Then J^{Ta) = {J^T)^ so that both and J^iT^) have 
their support contaimed in H . As the Killing form is on jV we can apply 
Lemma 8-1. It tells us that T is if the dimension of Qi is odd that is to say 
if — 1 is odd which means n is even. From now on we may assume that 
n is odd. We also get, on [g, g] 

{TJ{tX,,vy)) = \t\-^^'-'^/\TJ{X,,vy)) (1) 

and T = TT (in this case A = 1). 

8.3 Fourier transform on V ®V 

Let dv be a Haar measure on V and let dv* be the dual Haar measure on V* 
so that if 

i,{v*)= f f{v)T{-{v\v))dv 

Jv 

then 

f{v)^ I i^{v*)T{{v\v))dv* 

Jv* 
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On V ®V* we define the Fourier transform by 

f{w,w*)— I f{v,v*)T{—{v*,w) — {w*,v))dvdv* 
Jv®v* 

and the inversion formula is 

f{w,w*)— I f{v,v*)T{+{v*,w) + {w*,v))dvdv* 
Jv®v* 

For / e S{V e V*) let 

{J^f){v„V,)= [ f{v„V*)T{-{v*,V2))dv* 

Jv* 

be the partial Fourier transform. If g E G, put fg{v,v*) = f{gv, ^g^^v*). 
Then 

J'{fg){vi,V2) = \T>eig\{Tf){gvi,gv2) 

Moreover 

{J'f){vi,V2)^{Tf){-V2,V^) 

Let P e W[X] and 

ip{v,v*) = ip{v,v*)Ti-{v\PiX)v)) 

Then 

Let T be a distribution on q x V x V* , invariant under G, with support 
contained in the singular set and let S = J-'T be its partial Fourier transform 
on V*. The support of S is contained in J\f x V x V. For any polynomial P 
the function r((w*, P{X)v)) — 1 is equal to zero on the support of T, hence 
{t{{v*, P{X)v)) - 1)T = 0. This implies that 

{S,^iX,vi,V2 + PiX)v,)) = {S,ip) 

Suppose that the distribution T is skew with respect to the involutionno" 
and consider T the partial Fourier transform of T with respect to V (B V*. 
Then T is invariant under G and skew with respect to a, hence its support 
is contained in the singular set. In particular 

T{t{v*,v))T ^T, T{t{v*,v))f ^T, te¥ 
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On V ® V* the quadratic form Q{v,v*) — {v*,v) is non degenerate and 
7((5) = 1. It follows that T — T. This in turn implies that 

{S,(p{X, -V2,vi)) = {S,ip) 

Recall that if P is a polynomial and X a nilpotent endomorphism of V then 
P{X) is invcrtible if and only if P(0) 7^ in which case the inverse is R{X) 
for some polynomial R. Consider the group SL{2,¥[X]) of 2 by 2 matrices 
with coefficients polynomials in such a X and with determinant the identity 
endomorphism of V. The usual identities in SL{2) imply that this group is 
generated by the matrices 

(1 )' {p{x) 1) 

As a distribution on J\f x V x V the distribution S is invariant under the 
transformations 

{X, Vi, V2) ^ {X, -V2, Vi), {X, Vi, V2) ^ {X, Vi,V2 + PiX)vi) 

Therefore it is invariant under the transformations 

(X, + V2) ^ (X, A{X)vi + BiX)v2, CiX)v^ + DiX)v2) 

where A,B,C,D are polynomials such that A{X)D{X) - B{X)C{X) = 1. 
In particular, if P is a polynomial such that P(0) 7^ 0, it is invariant under 

(X,vi,V2) ^ (X,P{X)vi,P(X)-\2) 

This means that the distribution T satisfies the homogeneity condition 

{T,f{X,P{X)v,'P{X)v*) = |DetP(X)|-^(r,/) (2) 

We have to prove that a distribution T, having all the above properties, and 
(T— skew, is 0. 

8.4 Analysis on a fixed nilpotent orbit 

The nilpotent cone has a stratification given by the nilpotent orbits; each 
nilpotent orbit is stable by transposition. It would be enough to prove that, 
given a nilpotent orbit F, an invariant distribution S on F x V x V* , with 
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singular support and cr— skew is 0. Wc may assume that this distribution 
satisfies the homogeneity conditions (1) and (2) and is equal to its partial 
Fourier transform relative to V (BV*. 

If we fix X e rthen , using the Appendix, we can transfer the problem to 
V ®V*. Let C be the centrahzer of X in G; it is known to be unimodular. 
Given a T as above there exists a distribution S on V ® V* such that, for 
f eS{rxVx V*), 

{TJ)= [ {SJ{gXg-\gv,'g-'v*))dg 
Jg/c 

The distribution S is invariant under C. We have to transfer to S the prop- 
erties of T. 

Let (p e 5(7); the function g 1— > (p{gXg~^) as a function on G/C belongs 
to S{G/C) and this is a bijcction of 5(7) onto S{G/C). Let / e S{V © V*). 
The homogeneity condition (2) implies that for any polynomial P with non 
zero constant term 

/ vigXg-') {S, f(P(X)gv,' P{XYg-\*))dg 

J G/C 

= |DetP(X)|-^ / ^{gXg-^){S,f{gv,'g-\*))dg 

JG/C 



This is valid for any if therefore 

{SJ{P{X)gv,'P{XYg-'v*)) = \DeiP{X)\-' {SJ{gv,' g-'v*)) 

In a similar way one shows that S is equal to its Fourier transform S on 
V®V*. 

To transfer the homogeneity condition (1) we first remark that there 
exists a one parameter group 5{t) in G such that 5{f?jX5(f))^^ = tX. Indeed 
by Jacobson Morozov Theorem one can find in q two elements H and Y such 
that 

[H, X] = 2X, [H, Y] = -2y, [X, Y]^H 

The one parameter group with infinitesimal generator H/2 will do. We shall 
be more exphcit later on. Consider the adjoint representation of this TDS 
in 0. The subspace generated by the vectors having dominant weight is the 
Lie algebra c of the centrahzer G oi X. Also note that if I is an irreducible 



41 



component with dominant vector e then [X, e] = dim£ — 1. It follows that 
the trace of the restriction of adH to c is equal to the codimension of c in g 
that is to say to the dimension of G/C. 

As ad6{t)X = tX this implies that the determinant of the restriction of 
adS{t) to c is 

Det(ad5(i)|c) =t5dimG/C 

The group C is unimodular and normalized by ad5(i); let dc be a Haar 
measure. We have 

d{S{t)cS{t)-^) = \t\^'^'^^/^dc 

The Haar measure of G is invariant under ad5(i) therefore if /i is an invariant 
measure on G/G we get 

For / and ip as above 
/ ifitgXg-') {SJ{gv,'g-'v*))di,(g) 

J G/C 

= / ^{g5{t)X5{t)-'g-'){SJ{gv,'g-\*))d,i{g) 

J G/C 

= / ^{gX5g-'){SJ{g5{t)-\'g-'H{t)v*))dM 

J G/C 

From the homogeneity condition (1) we thus get 
Note that 

^(n^ - 1) - ^dimG/C = ^(dimC - 1) 

Finally we have to take care of the involution a. We know that there exists 
a hnear map s of F into V* such that *X = sXs~^. If T is cr— skew then a 
trivial computation gives for S the condition 

{SJ{s-'v\sv)) = -{SJ) 

To summarize we have to show that, for any fixed nilpotent matrix X, a 
distribution S onV (BV* which satisfies all the following conditions is 0. 
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(0) The distribution S is invariant under C, 

(1) For t e F* 

{sjmv,'s{t)-'v*)) = \t\-^('^^^-'Hs,f) 

(2) If P is a polynomial with non zero constant term then 

{SJiP{X)gv/P{X)'g-'v*)) = \DetP{X)\-'{S, f{gv,' g-'v*)) 

(3) The distribution S is equal to its Fourier transform 

(4) The support of S is contained in the singular set: 

Ex = {(v, V*) I for all p {v*, X^v) = 0} 

(5) The distribution 5* is " skew" . 

We proceed by induction on n. We may assume that n is odd; the case n — 1 
is done in section 4. In this case the condition (0) is enough to conclude. 

9 A descent method 

We keep the same notations and we want first to collect some informations 
on the centralizer C oi X. 

9.1 Jordan types and the structure of C 

In the nilpotent case a Jordan block of length r is the matrix of size r: 
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In a suitable basis X has a matrix which is a " direct sum" of Jordan blocks. 
The basis is not unique but the set of length of the blocks is. This defines 
the Jordan type of X. For example we shall say that X has Jordan type 
(1, 2, 2) if in some basis the matrix of X is the "direct sum" of one block of 
length 1 and two of length 2. 
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In general suppose that V = ViQ)V2 with X{Vi) C V^. Call Xj the restriction 
of X to Vi. Assume also that X2 is the sum of m Jordan blocks of the same 
size r. Then we may identify V2 with some vector space with W a vector 
space of dimension m in such a way that if V2 = {wi, . . . , Wr) is some element 
of V2, then X{v2) = {w2, ■ ■ ■ ,Wr,0). Thus the matrix of X2 is rir where 
the "1" now represent the identity map of W. Using the decomposition 
V — Vi®V2we have 

Let 

be an endomorphism of V^. It will commute with X if and only if 

aXi — Xia, Srir — rirS, Xi(5 — /Srir, ^Xi — rir^ 
The second condition means that 6 may be written as 



(5 = 





d2 
di 


dr \ 
. . . dr-l 




I 





rfi / 



with the di endomorphisms of W . 

Let us analyse the condition on (3. This map is a linear map from into 
Vi] call ui, . . . ,Ur its columns which are elements of Vi. The condition is 

XiUj — lij-i, for 2 < j <r, XiUi — 

To get such a /? we choose u E Vi and put Uj = X[^-'u. We must have 
XiUi — that is to say X{u — 0. 

The situation for 7 is similar except that we consider the rows and not the 
columns. 

From now on suppose X[~^ = 0. In other words r is the largest possible 
size for a Jordan block of X and X2 is the sum of all the Jordan blocks of 
size r. In this case we have X{u — for any choice of u and in fact ui — 0. 
Similarly the first row of 7 is arbitrary and the last row is always 0. 

Proposition 9.1 The determinant of c is 

Detc = Det{a)Det{diy 
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The proof is left to the reader. One can for example introduce a full Jordan 
basis for X, regroup the blocks of the same size and then proceed by induction 
on the number of sizes appearing in the Jordan type of X. In particular c is 
invertible, hence belongs to C, if and only if di and a are invertible. 
An element of V is written as {vi,wi, . . . , Wr) 

Proposition 9.2 The open set {v \ w,. 7^ 0} is an open orbit Q of C in V. 
This is clear from the description of the matrix of c. 

The involution a is defined using a basis (cj) of V, the dual basis (e*) and 
the hnear map u from V to V* such that u{ei) = e*. Then 

a{X,v,v*) = {u'^'Xu,u-^v*,u{v)) 

Let Wx be a bijective linear map of V onto V* such that *X = WxXw^^ 
and put s = uw^u. To be precise we choose a Jordan basis for X and if 
£1, . . . , £p is the part of the basis corresponding to one block and e^, . . . , £* 
the dual basis we take u{ei) — e* and wx{si) — Thus we may assume 

s = Wx and s s. A trivial computation shows that T is skew if and only 
if 

{SJ{s-'v\sv)) = -{SJ) 
9.2 Restriction to the open orbit 

Let O be an orbit of C in V. Let S be a distribution on {Ox V*) which is 
invariant under C and with support contained in Ex- Note that the orbit O 
is stable by P{X) for any polynomial P with non zero constant term because 
P{X) e C. Thus it make sense to assume that S satisfies condition (2). 
We arc going to prove that for some O, including the open one ft, such a 
distribution must be 0. 

We fix a point e E O and let D be the centralizer of e in C. Put 

^x,e^{v*\{X,e,v*)eE} 

Then Ex,e is the subspace of V* orthogonal to the subspace E{X, e) of V 
generated by X and e. There exists a distribution U on E^.e such that 

{U,aCd-'v*)) ^ AD{d){U,a{v*)) 

and 

{S,'ip{v,v*)) = (f {U,ip{ce,*c-\*))d^i{c) 

JC/D 
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We first use the homogeneity condition. As S is C— invariant we have, for P 
a polynomial with non zero constant term 

{S,^|;(P(X)-\PCX)v*) = {S,^(v,v*)) 

Hence 

{S,^{v,PCxfv*)) = {{S,i;{P{X)v,PCX)v*)) = \DetP{X)\-'{R,^{v,v*)) 
which implies 

{U,ip{PCxfv*)) = \DetP{X)\-'{U,^), ip e S{Sx,e) 

Note also that DetP(X) = P(0)".For our present purpose it will be enough 
to take for P the constant polynomial P{X) = t. 
Next we study the invariance condition under D. 

We shall need another assumption, we suppose that there exists a subspace 
W of V, which is invariant by X and such that V — W ® E{X,e). Put 
E — E{X, e) and consider the basis £i, . . . , £s of £■ defined by Sg-j — X^e, 
with s — (\\mE. Then we may represent X as 



X -- 

lide D then 

d 



Y 

Us 

a 

b 1 



with aY = Ya and Ugb = bY. In particular 
hence 

{U,^{t-'v*))^An{dt){U,^) 

We conclude that 

Aoidtf = n 

The equation Ugb — bY = is easy to solve. The condition is a condition on 
the rows of b, relative to the basis e* of E*, dual to the basis Sj. We choose 
w* e W* such that ^Y^w* = and define b by 

{e*bH)^CY^-\w*),w) 
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so that we obtain a commutative group isomorphic to Ker(*y''). 
To compute A^i we recall that 

Aoid) = |Det(ad(d))|-^ 

with ad the adjoint action on the Lie algebra d of D. Taking d = dt we 
deduce that 

Aoidt) = |i|dim{Ker*y^>) 

Finally, if the distribution S is non zero then 

n = 2dim(KerV) 

is even contrary to one of our hypothesis. 

The condition that E = E{X, e) is a direct factor for V viewed an F[X]— module 
is rather strong but is satisfied at least for the open orbit of C in V. 
Indeed start with a Jordan decomposition of X: 



( IT-r, \ 
r2 



X = 



"ri 

rir 



ri < r2 < ■ ■ • < r, 



s 



Put Vs — r and take for e the last vector of the basis. Then E{X, e) corre- 
sponds to the last block and W to the others. The orbit of e is F \ KerX''"^, 
the open one. 

The conclusion is that we may add to our assumption on S that its support 
is contained in Ker(X'^^) © V*. However we can reverse the role of V and 
V* and conclude that it is also contained in F ® Ker(*X''~^) so that it is 
finally contained in Ker(X''~^) ® Ker(*X^~^) Furthermore S — S (Fourier 
transform on V (BV*) Thus the distribution S is invariant by translations by 
elements of the subspace orthogonal to Ker(X''~^) © Ker(*X''~^), that is to 
say Im(X'-^) ® Im(*X'-^). Note that, for r > 2 

Im(X'-i) ® Im(*X'^-^) C Ker(X'-i) ® Ker(*X'^-^) 

Remark. — If r = 1, so X = and C — GL{V), then the distribution 
S has its support contained in {0} x {0} hence is a multiple of the Dirac 
measure. and must be equal to its Fourier transform, so it is 0. We may 
assume that r > 2. 
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9.3 The descent 



Assume X ^ . Let V ^ KerX""-^ /ImX'-K Then the dual space V* is 
Ker*X'^-Vlm*X'-^ The operator X is on ImX'-^ and X(KerX'^-i C 
KerX*""^ so that X defines a nilpotent endomorphism X' of V. Of course 
*X will give the transpose of X'. An element c of C will also define an endo- 
morphism of v. This gives an homomorphism 6' of C into the commutant 
C of X'. However 9 is not onto. 

To be more exphcit let us use the setup of 9-1. So V — Vi ® and an 
element of V is written as {vi,wi, . . . ,Wr)- The kernel of X^"^ is given by 
Wr = and the image of X^'^^ hj Vi = 0,W2 = = . . . = Wr = 0. We 
identify V with the subspace Wi = Wr = 0. Then one goes from V to V by 
forgetting the components relative to Wi and Wr- For example 

V rir J \ nr-2 J 

c=(; a..c'=.(c) = (;; f;) 

then a = a' and 6' is obtained from 6 by deleting the first and lasr row and 
columns so that if S is given hj Di, . . . ,dr then S' is given by cii, . . . , dr-2- 
The columns Uj of /3, which are elements of Vi are given by Uj — Xl~-'u with 
u arbitrary in Vi. Then P' = . . . ,Xiu). For C" a typical element P' 

is = (X[~^-u', . . . , -u') where -u' must be such that X[^'^u' = ( this is not 
automatic because, for some X, we may have 7^ 0). Therefore if we 

want 9 to be onto a necessary condition is that KerX[~^ is contained in the 
image of Xi. We know that Xl~^ = so that any Jordan block of Xi has a 
size at most equal to r — 1. For a block of size strictly smaller than r — 1 the 
contribution to the kernel of X^~^ is the full corresponding subspace which 
is not contained into the image. Thus the condition is satisfied if and only if 
Xi contains only blocks of size r — 1. We have to cases: either X has only 
blocks of size r or the only possible sizes are r and r — 1. 
We must also check 7; this gives the same condition. 

We now go back to S. From the result of the last subsection there exists a 
distribution S' on V such that, with obvious notations 

S — S' <^ dwi ® 5yj^ ® 5w* ® dw* 
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The Haar measures are normalized as the Fourier transform of the Dirac 

measures. 

We try to transfer the conditions on S to conditions on S' and to use induc- 
tion. It is obvious that if S satisfies conditions (3), (4) or (5) then the same 
is true for S'. The same is true for condition (2). This can be seen by a 
direct computation (note that DetP(X) = P(O)"' or by remarking that it is 

a consequence of conditions (3) and (4). 

Unfortunately in general S' is not invariant under the full commutant of X' 
but only under 0{C). In the first exceptionnal case where all the Jordan 
blocks have the same size, which must be odd because n is odd, we go down 
from size r to the size r — 2, keeping the full invariance and we can iterate 
until we arrive at size which means X = and we can conclude that S = 0. 
In the second case, starting from blocks of size r and r — 1 we go down to 
r — 1 and r — 2 and go on down to X = which again gives S = 0. 
In particular suppose that n = 3. Then the possible Jordan types for X are 

(1,1,1), (1,2), (3) 

and all three cases are settled. Therefore 

Proposition 9.3 Conjecture 2 is true for n < 4 

We will try to do a little better. 

Let us first deal with the behavior of condition (1) under descent. We have 
to choose d{t). On V2 we define it by 

and make any acceptable choice on Vi. The restriction to V is a possible 
choice for a one parameter group for X' . Condition (1) for 5" and the relation 
between S' and S immediately implies that 

{S'J{6{t)u,'6{t)u*)) = |t|™('-l)+|(dimC-l)^^j^ 

The degree of homogeneity is not the one we would want for S", that is to 
say |(dimC" — 1). There is a shift and this has to be kept in mind when 
using condition (1). 
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10 Some particular cases 

To use the descent we must first show that the distribution is on KerX''"-'^ x 
V*. If we have invariance under the full group C then KeicX^~^ is an orbit 
in C. However after going down once we will have invariance only under a 
subgroup Ci of C and KerX*"^^ is the union of a finite number of orbits. For 
each such orbit 7 we must prove that a distribution on 7 x V*, invariant under 
Ci, having singular support and satisfying the homogeneities conditions is 0. 
Also, in condition (1) we have to take the shift into account. This can be 
done in a limited number of low dimensional cases. 

We use the setup of section 2-1. We call Co the subgroup of C given by the 
conditions = 0, 7 = and we shall always assume that we have invariance 
under a subgroup Ci of C containing Cq. The one parameter group 6{t) 
will always be choosen inside Cq so that the precise choice will be irrelevant. 
Finally we shall take condition (1) as 

{s,f{6{t)v,' {tr'v*) = \tr{s,f) 

leaving the parameter fi free. Then for a given Ci we try to prove that S 
must be outside KerX''"^ x V*. If we use condition (1) then some values 
of n will have to be excluded. 

10.1 A simple remark for X = 

If n = 1 then any distribution on V (B V* which is invariant under GL{V) 
is symmetric with respect to a. Explicitly we identify V and V* to F and 
GL{V) to F*. The action is 

{t, {x,y)) ^ {tx,t-^y) 

and {{x,y)) = {y,x). 

Take any n and X — 0. Fix a basis (ei, . . . e„) of V; call (e*, . . . , e*) the dual 
basis. Let {xi, . . . , Xn) be the coordinates in V and (yi, . . . , y„) the 
coordinates in V*. 

Lemma 10.1 Let S be a distribution on V ®V*, invariant under the action 
of (F*)" acting by 

{{ti, ...,tn), {Xi, ...,Xn,yi,..., yn)) ^ {hXi, tnXn, t^^yi, t'^yn) 
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Then S is invariant by the involution 



a : {xi,...,Xn,yi,...,yn)^ {yi, ■ . . ,yn,xi, . . . , x^) 

We have to prove that for any / G S{V © V*) we have {S, a{f)) = {S, /). It 
is enough to do it for a product 

n 

f{x,y) = YlMxi,yi) 
1 

If we freeze all the fi except one, say fj then we get a distribution on F^, 
invariant under the action of F*. Therefore we can permute Xj and yj. This 
being true for all j we conclude that S is invariant by a. 
Take X of Jordan type (1, . . . , 1, 3 . . . , 3) ( with p times 1 and q times 3). 
Apply the descent method : X' = 0. The distribution S' is not invariant 
under the full GL{p + q) but is invariant under the subgroup GL{p) x GL{q). 
We may apply the Lemma to conclude that yS" = 0. In particular, for n = 5 
this takes care of the case (1, 1, 3) and for n = 7 of the case (1, 1, 1, 1, 3). 



10.2 The (l,...,l,r) case 

We are now going to use condition (1). Suppose that X is principal and 
choose a basis of V such that, in matrix form X — rir- Let 

/r-i ... \ 

n 
5{t)= ^ ^ ■■■ ^ 

V ... 1 ; 

let xi, . . . ,Xr be the coordinates in V. 

Lemma 10.2 Let S be a distribution on V (B V*, invariant under C, sup- 
ported by the singular set and which satisfies condition (1). If/i ^ — r(r — 1) /2 
then the support of S is contained in {v\xr — 0}xV*. 

Indeed {v e V\xr ^ is the open orbit of C in y. We may restrict the 
distribution S to {v\xr — 0} x V* . The centralizer of being trivial, there 
exists a distribution R on V* such that 

{SJ)= [ {R,f{cer,'c''v*)dc 
Jc 
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However the subspace generated by the vectors X'^e^ is V and the support 
of S is contained in the singular set so that R must be some multiple of 
the Dirac measure 93 i— > (^(0) on V*. Therefore, for some scalar A G C, the 
restriction of S is 

(5',/)= / f{cer,0)dc^X / /(V'a;jej,0) \xr\~''dxi . . . dxr 
Jc Jv^ 

and we get 

{SJ{5{t)v,'5{t)-^v*) = x[ f{J2t'~'^i(^i^^) \^r\~'dXi...dXr 



This implies the Lemma. 

Remark. — The proof remains valid for a distribution defined only on Xj. ^ 0. 

The conclusion is then that this distribution is 0. 

Now we take X of Jordan type (1, . . . , 1, r) with r > 2 and p times 1: 

Let Co be the "diagonal" subgroup of C: 

, aeGI/(p, F), brir = Urb 

Define 



( 1 













■ ° \ 
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V 














• 1 / 



Call ei, . . . , Cpj^r the basis of V and e^, . . . , e*_,_^ the dual basis. Let . . . , Xp^j. 
be the coordinates in V and 7/1, ... , Hpj^r the coordinates in V* . 



52 



Lemma 10.3 Let S be a distribution on V (B V*, invariant under Cq, sup- 
ported by the singular set and which satisfies condition (1). If jJ, ^ — r(r — 1)/2 
then the support of S is contained in {v\xp+r = 0} x V* . 

Put V ^ Vi®V2 with vi = ®l¥ei and V2 = e^+^Fe^. The Cq -orbit of 
Ci + ep^r is the open set in V defined by 7^ and Xp^r 7^ 0. The subspace 
generated be X and ei + e^+r is F(ep+r + ei) ©p^*!^""^ Fej. Let W be the 
subspace of V* orthogonal to this subspace of V; then 

W = {{yi, y2, ■ ■ ■ , yp, 0, . . . , -yi)} 

Let M be the centrahzer of ei + Cp+r- The group M is isomorphic to the 
centrahzer of ei in GL(p,¥). In particular it is not a unimodular group. On 
the other hand Co is unimodular. We have an integration formula 



f{c)dc= (p f{cm)drmdiJ,{c) 



Co JCo/M 

Here drm is a right Haar measure of M and djj, is a Cq— invariant linear form 
on the space of functions ip defined on Cq and such that ip{cm) = Am{'>ti)ip{c). 
Consider the restriction of S to Co(ei+ep+r) x V*- There exists a distribution 
R onW such that 

{R,iljCm-'w)) = AM{m){R,ilj) 

and 

{S, f)^ I {R, f{c{e^ + ep+r),' c-'w))di^{c) 

JCo/M 

We have 6{t){ei + Cp+r) = ^1 + ^p+r and is the identity on W and 
commutes with M, therefore 

{S,f{S{t)v,'S{t)-'v*)) = / {R,f{S{t)cS{t)-\ei + ep+r),HmcS{t)-'w))df,{c) 

JCn/M 



'Co/M 

dii{5{t)-^c5{t)) 



diJ,{c) 

and 



dii{5{t)-^c5{t)) _ d{5{t)-^c5{t)) 
dii{c) dc 
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A typical element of Cq is 





a 








... \ 







h 




br 












. . . br-l 


V 











b, J 



The Haar measure is 

dc — \bi\~^ dadbi . . .dbr 
and the maps c i— > 5{t)'~^c5{t) corresponds to 

(a, 6i, . . . , br) ^ {a, bi,t~^b2, t~''^^br) 

Therefore 



dii{6{t)-^cS{t)) 
dfi{c) 



It follows that the restriction of 5 is 0. The support of S is thus contained 
into the closed subset 



jF = e i/| Xp+r = 0}{v = + ^2 e I vi = 0} 

Let us restrict S to the open subset Xp+r oi On this subset we have 
vi = 0. Let / e S{vi) and /* e Then, on the distribution 

/r^(^,/(^)/rK)/2(^2*)) 

is invariant under the action of the subgroup GL{Vi) ~ GL(y*) of Co.This 
distribution must be a multiple of the Dirac measure at the origin and the 
support of the restriction of S is contained into V2 © V2*. If we view this 
restriction as a distribution on the open subset Xp+r 7^ of V2 © V2 we are 
exactly in the situation of the remark following the proof of Lemma 10-2. 
We conclude that the restriction of S to the subset Xp+r 7^ of ^ is which 
is exactly what we claimed. 

Now let us start again with X of type (!,...,!, r) and keep the same nota- 
tions. The starting value of // is 

/^ = ^((P+l)' + ^-2) 
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The support of the distribution S is contained into the set Xp+r — O.We now 
go down one step arriving at the type (l,...,l,r — 2). 
The new value of /i is 

1 1 

H' = H + r -1^ -{{p+lf + r -2) + r -1^ -{{p+lf + r - 4:) + r 

We have invariance under the group Co- As /i' is it is certainly not equal to 
the exceptionnal value and we can go down one more time. The value of /i 
will keep increasing, thus staying positive and we will keep invariance under 
Co. 

Eventually we will get the matrix, of size p or p + 1, keeping all the way 
invariance under at least the diagonal group and skew invariance . This 
implies that the distribution is 0. 

In particular it works for the type (1,4) which was the missing case forn = 5. 

Proposition 10.1 The general multiplicity one conjecture for GL{n) is true 
for n < 6. 

(this means up to restriction from GL{7) to GL{6)). 

10.3 The case n = 7 

The remaining types are: 

(1,1,2,3), ((1,2,4) (2,5) 

We shall fix a Jordan basis for X, say (ei, . . . , 67), the Jordan blocks being 

taken by increasing size. The dual basis is (e^, . . . , e^). The coordinates in V 
) and in V* they are {yi, . . . ,yr). The involution is built with 
the map s : 1— > 6(8 — i)*. 

Let us start with the (1,1,2,3) case. The distribution S satisfies all our 
usual conditions with the original value of fj, and invariance under the full 

centralizer C. Apply the descent method, in the original form, that is to say 
for the orbit of 67. Then we go down to the case (1, 1, 2, 1). The distribution 
5*1 that we get is invariant under the subgroup Ci of C described by the 
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picture 
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*/ 



Keeping the original numbering of the basis vector we see that Cie4 = {x^ ^ 
0} is an open orbit in the space W = Fei ©Fe2 ©Fes ©Fe4 ©Fee- We restrict 
Si to Cie4 X W*. we have Xe^ = 63, X'^e^ = 0. Let M be the isotropy 
subgroup of 64. Then a generic element of Ci is 



Cl 





02 

0,4 








di 





&1 

di 
C3 



0\ 







with 0104 — 0203 7^ 0, di 7^ 0, (is 7^ 0. The reductive part of Ci is obtained 
by taking bi — b2 — ci — C2 — d2 — b^ — cs — and the unipotent radical by 
Qi — a2 — di — d^ — 1, 02 = 03 = 0. Let 

Oi 02 
as 04 

The action of the reductive part onto the unipotent radical is given by 



(C1,C2) 

bi 

d2 
C3 

^3 



di^ 



di{ci,C2)a ^ 

1-^ (^2 

bsdi/ ds 



The Haar measure of the unipotent radical is dbidb2dbsdcidc2dc3d{d2) and 
is invariant under the action of the reductive part. It follows that Ci is 
unimodular. 

Now we look at M so wc take &i = 62 = = C3 = and di — 1. The 
reductive part is obtained by adding the conditions bs — Ci — C2 — while 
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the unipotent radical corresponds to a = 0, — 1. The adjoint action of 
the reductive part onto the unipotent radical is 

63 1-^ bs/ds, (ci,C2) ^ (ci,C2)a"^ 

The Haar measure of this radical is du — db3dcidc2 hence, for h in the 
reductive part H 

dz\~^\DQia\~^ 



d{huh-^) 



du 

Therefore the right Haar measure of M is given by 

/ f{m)drm— I f {au)\d2,\~^\Deia\~^ dhdu 

JM JhxU 

and the modulus of M is AM{hu) = |(i3||Deta| 

We restrict our distribution to €064 x W*. There exists a distribution R 
on W* such that 

(i?,V'Cm-V)) = AMM(i?,V') 

and 

/ {R,f(ce,,'c-'w*))di,(c) 

JCi/M 

The support of 5*1 is singular so that the support of R must be contained 
into the subset ys — — 0. The semi-invariance under M means that 

{R,'4^{{yi,y-2)a,d^y^) = |d3||Deta|(i?, V') 

In particular freezing we obtain a distribution Ri on (^1,^/2) such that 
{Ri,(p{{yi,y2)a)) = |Deta|(i?i, 99). The only such distribution is so that 
R — and finally the restriction of Si is 0. 
We could also use the homogeneity condition 

{R,ip{t'w*)) = \t\-'{R,i;) 

Now in M we choose a — tld and d^ — t. By the semi-invariance 

W(t(i/i,i/2),t|/6)) = W{R,i^) 

Remark. — The point here is that there exists an clement of the centralizer 
M of the base point e in \^ which is multiplication by t on E[X,v)-^. This 
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was already the case in our original descent method. It is unfortunately not 
true in general but could be true in a number of cases. 
The support of 5*1 is contained into the subset = 0. Then using again 
the Fourier transform and a we can go down to the Jordan type (1, 1, 1) and 
an invariance groupe Co isomorphic to GL{2) x GL(1). This is enough to 
conclude that our new distribution S2 is and consequently that our original 
distribution 5 is 0. 

The other 2 cases may be settled in a similar way We omit the details. 
Then: 

Theorem 10.1 The general multiplicity one conjecture for GL{n) is true 
for n < 8. 

(this means up to restriction from GL{9) to GL{8)). 

Using the same kind of elementary computations more cases can be done. 
For example if there is only two possible sizes for the Jordan blocks ( with 
or without multiplicity) and if we assume invariance under Co, then except 
for some values of /i the descent will be justified. If we take n = 9 this takes 
care of all the types with two possible sizes and we are left with 7 cases like 
(1,2, 6), (1,2, 2, 4)...). 

Note that we are only using invariance under some subgroup Ci, the singu- 
larity of the support and the homogeneity conditions. Then, in each case, if 
r is an orbit of Ci in V contained in V \ KerX^~^ we proved that on F x V^* 
the distribution must be 0. However it is not difficult to show that in gen- 
eral such F X y* do carry non zero distributions with these properties (this 
already happens with a 3 block situation and Ci = Cq). The problem is then 
to prove that such distributions can not extend to invariant skew distribution 
on V (B V* . In some sense the case n = 1 is already typical. 

11 Another approach 

On V®V* we have the non degenerate quadratic form Q{v, v*) = {v*, v) and 
therefore the dual pair 0{Q) x SL{2) acting on S{V®V*). The distribution 
S is invariant under SL(2). Therefore we may consider it as a linear form on 
the covariant space which is the space of the minimal representation of the 
orthogonal group. The group C acts on V (BV* and this is an imbedding of 
C into 0{Q). The restriction of the minimal representation to C turns out 
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to be a finite sum of inequivalent irreducible members of some degenerate 
principal serie representations. This allows us to reformulate our problem in 

a more representation theoretic way. 

The quadratic form Q does not depend on X so we go back to our distribution 
T which we view as a linear form on S{J\f ®V ®V*). 
Consider the partial Fourier transform relative to V*: 

f^l f{X,V,,v'')T{-{v*,V2)dv* 

Jv* 

As before r is a non trivial additive character of F and the Haar measures 
dv and dv* are dual measures. 

Let R be the partial Fourier transform of T. It has the following properties. 

(0) It is Invariant under G: 

{RJ{gXg~\gv,,gv2)) = \Detg\-\RJ) 

(1) As a distribution on [g, g]©l/®y it is equal to its partial Fourier transform 
relative to [q, g] and 

{R,f{tX,v,,V2)) = \t\-^''-'^/'{R,f) 

(2) li A, B,C, D are 4 polynomials in one variable, such that AD — BC — 1 
then 

{R, fix, A{X)vi + B{X)v2, C{X)vi + D{X)v2)) = {R, f) 
(see section 1-3). 

For the involution we choose a basis (cj) of V with dual basis (e|). Let (xj) 
be the coordinates in V and [x*) the coordinates in V*. Let u, a linear map 
from V onto V* be defined by u{ei) — e*. Then we may take 

a{X,v,v*) = {u-^^Xu,u-\v*),u{v)) 

On F let dx be the measure self dual with respect to r. We take dv = 
dxi . . . dxn and dv* — dxl . . . dx^. These measures are self dual and the 
image of dv by it is di'*. If 

T{X, Vl, V^) = / f{X, Vi, V2) t{-{vI, Vi) - {V2, V2))dVidV2 
JVxV 

then, after partial Fourier transform on V* the involution a becomes 
/(X, vi, V2) ^ J^f{X, u{v2), -u{vi)) 
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Therefore T is skew if and only if it! is skew: 

{R,Tf{X,u{v2).-u{v,))) = -{R,fiX,v^,V2)) 

Note that (2) imphes in particular that {R, f{X, —V2, Vi)) — {R, f{X, Vi, V2)) 
so that the above condition may be rewritten as 

{R,J^f{X,u{v,),u{v2))) = -{RJ{X,V,,V2)) 

In particular the distribution R is invariant under SL{2, F) acting on V (BV 
through 

/ -ui \ ^ / a b\ f vi \ _ f avi + bv2\ 

\V2 J \C d J \V2 J \CVi+dV2j 

t and acting trivially on g. 

The center Z of G acts trivially on g and R is semi-invariant under Z: 

{R,f(X,Xv„Xv2))^\XnR,f) 

We shall first find all linear forms L on S{V © V) which have the above 
invariance. 

We stratify V ®V using the rank of (vi, V2); the subset of (vi, ^2) with rank 
r (resp at most r) is denoted by {V © V)=r (resp V © V)<r. They are all 
locally closed. Then for r = 1 and r = 2 we have short exact sequences 

S{{V © V)=r) S{{V © V)<r) S{{V © V)<r-i) 

Now the stratification is stable for the action of the group SL{2,¥) x Z, so 
working in the category of smooth modules we have long exact homology 
sequence 

H,{S{{V © V)<r-i)) Ho{S{{V © V)=r)) — Ho{S{{V © V)<r)) 

The group G commutes with SL{2) x Z hence the above exact sequences are 
exact sequences of G— modules. 

Let us consider first {V © V)=q = {(0,0)}. Then S{{V © V)=o) = C. The 
group SL{2) acts trivially and the group Z acts as 

/(0,0)^/(^-^(0,0))|Det^|-^ 
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that is to say by the character |Det2;|~^. The action of G is also given by this 

character. 

There can be no hnear form with the required semi-invariance so i7o((^ ® 
V)=o) = (0). 

Let us next consider {V ® V)=i. If {vi,V2) ^ {V ® V)=i then the subspace 
generated by Vi and V2 is an element of the projective space P(y). hence we 
have a map p from {V © V)=i onto the projective space. We use Bernstein 
localization principle . The group is SL{2) x Z; each fiber of p is stable. 
We are looking for distributions on (V © V)=i which under the action of 
SL{2) X Z transform by 

{LJ{\{av^ + hv2),\{cv^ + dv2))) = W{LJ) 

where 

ad — be — 1, and z — /ild 

This is clearly a closed set of distributions, stable under multiplication by 
locally constant functions which factor through p. 

Then in order to prove that the only such distribution is it is enough to prove 
it on each fiber. So let us choose a point in F{V) and one of its representative 
1) in y so that {v, 0) is a point in the fiber. The group SL{2) x Z, acting on 
V ®V is the same thing as the group GL(2)+, the exponent + meaning that 
the determinant is a square. Then 

a b\ f v\ _ f av 
c d ) \Q ) ~ \ cv 

so that the group is transitive on the orbit. The isotropy subgroup is the 
group of matrices 



1 h 
d 



d e F*2 



The map 

{x,y) ^ {xv,yv) 

is an homeomorphism of \ (0, 0) onto the orbit. The distribution must be 
invariant under SL{2) hence proportionnal to dxdy. Under the action of Z 
the measure dxdy satisfies 



F2 



f{lJ,x,iJ,y)dxdy = \iJ,\ f{x,y)dxdy 
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We may assume that n> 3 and even that n is odd so this is not compatible 
with the required scmi-invariance, except if the distribution is which means 
that Ho{S{V © V)=i)) = (0). The sequence 

Ho(S{(V © VU)) Ho{S{(V © y)<i)) Ho(S((V © V)=o)) 

is exact, the first and third terms are 0, hence the second term is also 0; there 
is no distribution with the required invariance supported on the "singular set" 
rank{vi,V2) < 2. 

The next thing to do is to find the distributions on S{V (BV)=2- We first find 
the covariant space for the action of SL{2). Consider the map (f i, V2) ^ ^1^2 
from V(BV into '^V. it is immediate to see that SL{2) is simply transitive on 
the fibers. Each fiber tis homeomorphic to SL(2). Fix a basis ei, . . . , e„ of 
Call H2 the isotropy subgroup in G = GL{n) of 6162- Then G acts transitively 
on the quotient space SL{2)\{V(BV)=2 and this space is homeorphic to G/H2. 
if, for / e S{{V © V)=2) we put 




then / I— > </? is a map onto S{G/ H2) and the transpose is a one to one map 
from S'{G/H) onto the space of distributions invariants under SL{2). This 
means that the space of covariants for the group SL{2) is precisely S{G/H2). 
The group G acts by 

g(p{x) = \Detg\'^(p{g~'^x) 
It will be convenient to define 

ip{x) — ip{x)\Detx\ 

The function is such that ip{xh2) — \Deth2\ip{x) and, on ip, the group G 
acts by left translations. 

The group H2 is almost a maximal parabolic subgroup. In fact an element 
/12 e H2 is written as 




with mi e SL{2),'m2 G GL{n — 2) and u an arbitrary matrix with 2 rows 
and n — 2 columns. If we simply impose mi e GL{2) we obtain a maximal 
parabolic subgroup P2- 
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For any multiplicative character x of 1^* the representation vr^ of G is defined 
as follows. The space E-^ of vr^ is the space of functions w from G to C, 
locally constant and such that 

w{gp2) = If (fi')|Detm2||Detmi|"(""^^/^X~^(Detmi) 

The group acts by left translations. This representation belongs to the degen- 
erate principal serie associated to P2 and to the unitary degenerate principal 
serie if the character x is unitary. The central character of vr^ is 

Aid ^ x-\t) 

if we want this character to be trivial we must have = 1 which gives only 
a finite number of possibilities 

Let df be the diagonal matrix with diagonal coefficients (t, 1, 1, . . . , 1). Then 

for ip as above we put 

Jw* 

Then ipy. E E^. 

The distribution L on © l^) = 2 is given by a distribution, which we still 
call L on G/H2- We are left with the condition 

It is easy to prove that if, for all unitary characters x such that = 1, we 
have ■0^ = then (L, '0) = 0. 
It follows that 

Proposition 11.1 The representation vr^ are irreducible and two by two in- 
equivalent 

Rubenthaler [10] Theorem 5-4 page 481. The non equivalence is Theorem 
5-8 of the same reference. 

We will study the projection map from S(y © V)=2 onto E^. This map is 
given by 

f^{g) = \Det{g)\J^^^^^f{x (^^^J ) )x(x)|Det(a;)r/^rfa; 
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More generally for any complex number s put 



Put 



then 



X — 



Xii Xi^2 
X2I X22 



fxAg) = |Det(5()| / /(^(xiici + Xi2e2),£/(a;2iei + X22e2))x(a;iiX22 - a;i2a:;2i) 

JF4 



Fii^22 — a;i2a;2i 



s+n/2 dxiid.ri2d.r2idx22 

13^112^22 ~ 3;i2X2lP 



A priori we take / with support in a;iia;22 — a;i2a;2i 7^ so that the integral is 
always convergent. If wc take any f & S{y ®V) then the integral converges 
for 3?e(s) +n/2 > 1. In particular, if n > 3 the integral will be convergent for 
s = 0. For such / the mapping f ^ f\ has the same invariance properties 
as before so that this mapping may be considered as a mapping from the 
covariant space HQ^SiV ® V)) onto extending the map from HQ{S{y ® 
V)=2) onto E-^. This shows that: 

Proposition 11.2 For the action of SL{2,¥ x Z the covariant space of 
S{V®V)is 

Ho{S{V © V)) « Ho{S{V © V)=2) ~ 0^x 

At this stage our problem is to study the behaviour under the involution a 

of G— invariant linear forms on S{f/) ® {(BE^). 

On V (B V the involution a is essentially the Fourier transform. We now 
compute the corresponding involution on the E^ 

To define the involution a we choose the map u :V ^V* defined by u{ei) — 
e*. If / e S{V © V) then 

(^{f){vi,V2) = f{u{v2),-u{vi)) 

To simplify the notations we will, until the end of the section, identify V and 
V* with F" so that u becomes the identity operator. . . In © we call the 



64 



coordinates Xi, . . . , x„ for the first copy and yi, . . . ,yn for the second one. 
On © consider the quadratic form xi?/2 — 2:22/1 and the corresponding 
bihnear form 

X12 + l2/2 - X21 - 22/1 

For (fi G S{¥^) let 

^(1, 2, 1,2)= / v{xi,X2, 2/1, 2/2)^(2:12 + i2/2 - 2:21 - 2yi)dxidx2dyidy2 



'4 



If (f is the Fourier transform defined with the usual bilinear form (xn +X22 + 
yu + 2/22) then 

^(1,2,1,2) = ^(2,-1,-2,1) 
The additive character r is fixed . We may assume that it is of order 0. 
For G 5(F'^) and x unitary put 

, dxidx2dyidy2 



|2 



Z^ix,s)= / v^(a;i,a;2,2/i,y2)x(a;i2/2 - a;22/i)|2:i2/2 - a;22/i| , 

Jf4 |a;i2/2 - a;22/i| 

In [9]we proved that this integral is convergent for 3?e(s) > 1, extends to a 
meromorphic function of s and satisfies a functionnal equation 

The factor p{x, s) is defined in[9] We take x^ = 1- Then Proposition 3-1 of 
f9] gives explicitly the singularities of Z^. In particular the only possible real 
poles are and 1. We want to take s = n/2 and s = 2 — n/2 with n odd so 
we will have no problem with the poles. Finally note that = Z-x. 
Let fgivi,V2) = fi9Vi,gv2). Then 

fxAa) = P^^{9)\ / /<;(a;i,a;2,0, ... ,0,2/1,2/2,0, ... ,0)x(xi2/2 - X22/1) 

is+n/2 dxidx2dyidy2 



\x1y2 - X2yi\ 



\x1y2 - X2yi 



2 



Using the functional equation 

f^^sig) = |Det((7)rV(-l)p(x, s + n/2)pix, s + n/2- 1) 

if)tg-iiXi,X2,Xs, . . . ,X„,2/l,2/2,l/3, •••,!/«) 



]p4 Jf2n-4 

dxidx2dyidy2 
^' 'a;i2/2 - a;22/iP 



x(xi2/2 -X22/1) 1 2^12/2 -2:22/1 P ' "/^(JJrfxjd2/, 



3 
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In this formula the integral is convergent for 3fJe(s) < l — n/2 and f^^g is the 
analytic continuation. Let N be the subgroup of matrices 



n 



h 
c 





In-2 



We put 



/ 



In the above double integral we may integrate over the Zariski open set 
^iVi — X2yi 7^ and then make the change of variables 

Xj = xij + X2j j = 3, . . . , n 
Vj = yij + y2j j ^^,---n 



Note that 



n{xie-i + X2e2) = XiCi + ^262 + ^ xij + X2j 



and a similar formula for n{y\ei + ^262)- Thus 

f^,s{g) = |Det(^)|-ix(-l)p(x, s + n/2)p(x, s + n/2 - 1) 



_(/)*g-ln(^l' ^2,0,..., 0, 1/1, 1/2,0,..., 0) 



X(xiy2 - X2yi) \x1y2 - X2yi \ ^+"/^((/n) 



dxidx2dyidy2 
\x1y2 - X2yi\'^ 



This integral is convergent for 3fJe(s) < l — n/2 and is then equal to 
/x,.(^)=x(-lKx,s + ^/2)p(x,s + n/2-l) / f^,_,Cg-'n)dji 



N 



We want to take s = and without surprise the integral is divergent. Also 
we may replace / by a{f) (they are equal up to the action of the usual Weyl 
group element of 5'L(2) and this action dies in the integral so 

fxAg) = X(-1)P(X, s + n/2)p{x, s + n/2- 1) f_a{f)^,.sC9~'n)dn 
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Finally replace / by a{f) 

<7{f)xM = X(-1)P(X, s + n/2)p(x, s + n/2- 1) f_f^^_,{'g-^n)(M 

JN 

If s = and = 1 we have obtained a linear involution o"^ on such that 
a^TT^{g) = Ti^i^ g~^)a^. The representation tt^ is equivalent to its contragre- 
dient and a^^ is one of the two involutive intertwinning maps. 
Our distribution R may be viewed as a G— invariant linear form on S{H) (8) 
(©£'^). For each x we restrict it to S{M) ® E^. It is skew symmetric: 

(i?, © aje)) = -(i?, /(X) © e) 

Note that because of the homogeneity with respect to X we can replace *X 
by — *X. We have to prove that such R arc 0. 

This can be reformulated in various ways. First one can fix a nilpotent orbit 
in Q and some element X of this orbit. Define wx as in section ??? and let 
C be the centralizcr of X in G. Then it would be enough to prove that a 
linear form L on E-^ which is invariant under C and skew: 

^(^x(e)) = -L{'K^{wx)e) 

is 0. 

Now consider the outer automorphism of G defined by g h- > ^g~^. Let 
G — 2,2 ^ G he the corresponding semi-direct product. Thus if Z2 = 
{!,/«} then *g~^ — KgK~^ and the group law is given by {l,g){K,g') — 
{k,* g'^g'), {n,g){l,g') = {n.gg'). A representation tt of G in some complex 
vector space E is specified by its restriction to G and by a linear involution 
7f(fi;) such that Tf{n)'n{g) = Ti{^g~^)'W{K.). 

We obtain an extension 7f^ of tt^ to G by defining 7f^(«;) = cr^ and an extension 
V of the action 1/ of G in >S(jV) by letting k, acts through f{X) 1— > f{—*X). 
Then our distribution i? gives a G— intertwinning map ^4 between S{J\f) and 
the smooth dual E^ of The skew symmetry now means that 

Mx) = -7f*(«;)A 
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Part II 

Orthogonal and Unitary Groups 



12 . Introduction 

Let F be a local field non archimedean, of caracteristic 0. Let D be a either 
F or a quadratic extension of F . If x e D then x is the conjugate of x if 
D 7^ F and is equal to x if D = F. 

Let be a vector space over D of finite dimension n + 1 ^2. Let (., .) be a 
non degenerate hermitian form on W . This form is bi- additive and 

(dty, d'w') — d d'{w, w'), (w', w) = (w, w') 

Given a D— linear map u from W into itself, its adjoint u* is defined by the 
usual formula 

{u{w),w') = {w,u*{w')) 

Choose a vector e in W such that (e, e) 7^ 0; let U = e3 and V = U^, the 
orthogonal complement. Then V has dimension n and the restriction of the 
hermitian form to V is non degenerate. 

Let M be the unitary group of W that is to say the group of all D— linear maps 
m of W into itself which preserve the hermitian form or equivalently such 
that mm* = 1. Let G be the unitary group of V. With the p-adic topology 
both groups are of type Ictd ( locally compact, totally discontinuous and 
countable at infinity) . They are reductive groups of classical type. 
The group G is naturally imbedded into M. Our goal is to show that the 
following conjecture follows from the conjectures of Part I 

Conjecture 1' If n (resp p) is an irreducible admissible representation of M 
(resp ofG) then 

dim (HomG(7r|M,p)) < 1 



Choose a basis ei, . . . e„ ofV such that (cj, Cj) e F. For 

n 

W = XqC + ^2 
1 
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put 

n 

W = XqC + Xi Cj 

1 

If M is a ©—linear map from W into itself, let u be defined by 

u{w) = u(w) 

Let a be the anti-involution a{m) = of M; we will show that conjecture 
1' is a consequence of 

Conjecture 2' A distribution on M which is invariant under the adjoint 
action of G is invariant under a. 

The involution a depends on the choice of the basis of V. However, changing 
the choice of the basis will just replace a by gcrg^^ for some g E G so that 
the action on the space of invariant distributions does not depend on this 
choice. 

The proof follows exactly the same path as in Part I. There are two mains 
differences. On one hand the Levi components of some of the parabolic sub- 
groups of G may not be of the orthogonal or unitary type; some components 
of type GL appear and this is why we need the ( conjectural) results of Part 
I. On the other hand the singular set is much simpler than in Part I. It has a 
natural stratification stable by the involution and such that on each strata an 
inductive argument works. This does not seem to be the case for the general 
linear group. 



13 . Conjecture 2' implies Conjecture 1' 

In Chapter 4 of [B] the following result is proved. Choose 6 G GLf{W) such 
that {Sw,Sw') = {w',w). If vr is an irreducible admissible representation of 
M, let 71* be its smooth contragredient and define vr^ by 

Then and vr* are equivalent. We choose 5 = 1 in the orthogonal case 
D = F. In the unitary case, fix an orthogonal basis of W, say ei, . . . , e„+i, 
such that 62, ... , Cn+i is a basis of V; put (e,, Cj) = a,. Then 

Xid, ^ yjCj) = ^ ttiXiyl 
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Define 5 by 
Note that 5^ = 1. 

Let be the space of tt. Then, up to equivalence tt*, is the representation 
m I— > 7r(5m5~^). If p is an admissible irreducible representation of G in a 

vector space Ep then an element A of Hom (^t^*q, pj is a linear map from E^^ 

into Ep such that 

^7r(550=7r(^)A g^G 

In turn the contragredient p* of p is equivalent to the representation g i— >■ 
p{6g6^^) in .Ep. Then an element B of Hom (ttig, p*) is a linear map from E^^ 
into E'p such that 

B7rig)=p{6g6-')B, geG 
As 5^ = 1 the conditions on A and B are the same: 

Hom (7r*(3, p) ^ Hom (ttig, p*) 

However, assuming Conjecture 2', by Corollary 2-1 we have 

dim^Hom (7r*Q, p)^ x dim^Hom (ttig, p*)^ < 1 

so that both dimensions are or 1. replacing p by p* we get Conjecture 1'. 
From now on we forget about Conjecture 1'. 



14 . A partial linearization 

The group G acts on itself by the adjoint action and on V by the natu- 
ral action so it acts on G x and we may consider the space of invariant 
distributions S{G x V)' onGxV. Put 

(^{{9,v)) = {g~\-v) 

It is an involution. 

Conjecture 3' A distribution on G x V which is invariant under G is in- 
variant under a. 



70 



We claim that Conjecture 3' implies Conjecture 2'. Assume Conjecture 3' 
and apply it to M acting on M x W . Recall that W = U ®V with U a non 
isotropic subspace of dimension 1. Let e be a non zero element of U . Let 

Y ^ {{m,w) e M X W \ {w ,w) ^ (e, e) } 

It is a closed subset, invariant under M and under a. Any distribution on Y 
which is invariant under M is thus invariant under a. 

Now let X C F be the set of all pairs (m, e). By Witt's theorem MX = Y. 
If (m, e) e X and if for some m' & M one has m'{m, e) e X then m'e = e 
which is equivalent to m' e G and then m'X — X. We are in position to 
use a Frobenius type descent as described in the Appendix. In our case both 
groups M and H are unimodular. Choose a Haar measure on M/H. Let S 
be a distribution on X which is invariant under G. Then define a distribution 
9{S) on Y by 

{0{S)J)= f {SJ{mx))dm 

J M/G 

This is a distribution on F, invariant under M hence invariant under a. The 
map S I— s> ^(5*) is a bijection between the space of G— invariant distributions 
on X and the space of M— invariant distributions on Y . Let us compute 
e{a{S)). 



{e{a{S)),f) = [ {a{S),f{m{m',e)))dm 

J M/G 



'M/G 

= f (S,/((m.K)m-',me))d™ 

Jm/g 

— / {S,a{f){a{m~^)m'a{m), —me) dm 
Jm/g 

Now the map m i— > (7(m~^) fixes G and defines an involutive automorphism 
of M/G. In the above integral we change m into a{m~^) — m: 

JM/G 

The linear map u : w ^ —w belongs to the center of M hence, changing m 
into um we finally get 

{e{a{S)),f) = I {S,a{f){mm'm-\me))dm 
Jm/g 

= {0{S),a{f)) 
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As 9{S) is invariant under a this implies that 9{S) = 6{a{S)) hence S = <j{S). 
But X M so we do get Conjecture 2'. 

Let Q be the Lie algebra of G.The group G acts on g x V". On gxV consider 
the involution a given by a{x,v) = {—x,—v). To prove Conjecture 3' we 
shall first show that, in turn, it is implied by 

Conjecture 4' Any distribution on g x V which is invariant under G is 
invariant under a. 

This is done using Harish- Chandra descent method. This means that we have 
to consider the Levi factors of the parabolic subgroups of G. This involves 
unitary groups of lower ranks and, unfortunately, groups of type GL{n) over 
finite extensions of the base field. This is why the results of this second part 
are subject to the validity of the conjectures of the first part. 
We assume that Conjecture 4 of the first part has been proved and then 
proceed by induction on the dimension of V . 

15 . The case n = 1 

In the orthogonal case D = F, the group G is ±Id. It acts trivially on itself 
and on its Lie algebra ( which is reduced to (0)). On 1/ = F, the involution 
is —Id so both Conjecture 3' and 4' are tautological. 

In the unitary case D is quadratic extension of F. The group G is the group 
of g & G such that g'g = 1; it acts trivially on itself and on its Lie algebra 
and by multiplication on W = 3. On G and g the involution is trivial and 
on D it is (i I— >^ d. The orbits of G in G x V are stable by a and furthermore 
if we denote by ■j{g) the action of G on G x then 'j{g)a = crjig). A 
classical result of Bernstein-Zelevinsky ( see [6], chapter 4, page 91) asserts 
that Conjecture 3' is true. The same argument works for the Lie algebra, 
giving Conjecture 4'. 

16 . Harish- Chandra's descent 

Let us go back to the general situation. Let a G G, semi-simple; we want 
to describe its centralizer in G. View a as a ©—linear endomorphism of V 
and call P its minimal polynomial. Then, as a is semi-simple, P decomposes 
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into irreducible factors P = Pi . . .P^ two by two relatively prime. Let = 
Keri-'j(a) so that V = ©V^. Any element x which commutes with a will 
satisfy xVi C Vi for each i. For 

R{T) = do + • • • + dmT"", dodm ^ 
let _ 

Then, from aa* = 1 we obtain, if m is the degree of P 

{P{a)v,v') = {v,a-'''P*{a)v') 

( note that the constant term of P can not be because a is invertible). It 
follows that P*{a) = so that P* is proportional to P. Now P* = P^ . . . P* 
hence there exists a bijection r from {1, 2, . . . , r} onto itself such that P* is 
proportional to Pr(i)- Let be the degree of Pj. Then, for some non zero 
constant c 

= {Pi{a)vi, Vj) = {vi, a-'^'P*{a)vj) = c{vi, a'""' Pr(i){a)vj) , Vi e Vi, Vj e Vj 
We have two possibilities. 

Case 1: T(i) = i. The space Vi is orthogonal to Vj for j ^ i; the restriction of 
the hermitian form to Vi is non degenerate. Let = D[T]/ (Pj) and consider 

as a vector space over Dj through the action {R(T),v) i-^ R{a)v. As a|y. 
is invertible, T is invertible modulo (Pi)', choose U such that TU = 1 modulo 
(Pi). Let Tj be the semi-linear involution of Dj, as an algebra over D: 

J2 djT^ ^ Yl modulo {Pi) 

If Pj is of degree one, that is to say if the restriction of a to is a scalar 
operator then Dj D. Any linear map Xi from Vi into itself commutes with 
flj = ci\Vi- In this case we define Gi to be the unitary group for the restriction 
to Vi of our original hermitian form. We also put Fj = F. 
If Pi has degree at least 2, let Fj be the subfield of fixed points for Tj. It is a 
finite extension of F and Dj is a quadratic extension of Fj. Let L be a non 
zero D— linear form on Dj.Then any D— linear form £ on Dj may be written 
as (i L{\d) for some unique A G ©j. We claim that we can choose A 7^ 
in such a way that i{Ti{d)) = Ti{i{d)) for all d G Dj. Indeed dy^ Ti {L[Ti{d))) 
is D— linear form so there exists a G such that, for all 0? G Dj 

ri{L{Ti{d)))^L{ad) 
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One gets L(d) — L(aTi{a)d), hence ari{a) = 1. By Hilbcrt's theorem 90, 
there exists ^ G Dj such that a — fj,/Ti{ii). Then i : d L{iid) has the 
required property. 

If v,v' G Vi then d h- >■ {d{a)v,v') is D— hnear map on Dj hence there exists 
S{v, v') G Bi such that 

{d{a)v,v') =£{dS{v,v')) 

One checks that 5" is a non degenerate hermitian form on as a vector space 
over Dj, a quadratic extension of Fj. 

Also a ©—hnear map Xi from Vi into itself commutes with if and only if 
it is Dj-lincar and it is unitary with respect to our original hermitian form if 
and only if it is unitary with respect to S. So in this case we call Gi is the 
unitary group of S. It does not depend upon the choice of i. As no confusion 
may arise, for A G Dj we define A = rj(A). 

We have an involution a on G x V well defined as soon we have chosen a 
basis of Vover D such that the hermitian form(., .) has, relative to this basis, 
a matrix with coefficients in F. To build such a basis we are going to pick a 
suitable basis of Vi for each i. 

\\c stay with Case 1. Choose a basis {^k) of Vi over ©j such that S{^k, Ck') G 
Fj.Then, for f = ^ Afc.^^ we define v = )^kC,k- If / G EndiD)i(Vi) we define / 
by f{v) = f{v). Then, on {Gi, Vi) we have the involution 

CTi- {9i,Vi) ^ {g'i~^,-vl) 

and Qi being the Lie algebra of Gi, on Qi x Vi 

Gi'. {Xi,Vi)^ {-Xi,-vl) 

Now let {ur) be a basis of Fj over F; it is also a basis of over D. Then 
[ur, Ck) is a basis of Vi over D and, owing to our choice of £ 

{Urik,Ur'ik') = £{S{Ur^k,Ur'^k') = HUrUr'S{^k,^k'))) 
= i {UrUr'S{^k', Ck))) = {Ur'Ck', UrCk) 

Thus to define the global involution a, for the Vi part, we may choose this ba- 
sis. Note also, for f = ^ yr.kUr^k G Vi with l/r^k G D we have V = J2 yr~kUrCk- 

If we consider Gi x Vi as imbedded, in the obvious way, inside G x V then 
on this subspace the involution a coincides with the involution cTj. 
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Case 2. Suppose now that j = rii) ^ i. Then V^© V, is orthogonal to for 

k ^ i.j and the restriction of the hermitian form to Vi(BVj is non degenerate, 
both Vi and Vj being totaUy isotropic subspaces. Choose an inverse U of T 
modulo Pj. Then for any P G ©[T] 

{P{a)vi,Vj) = {vi,P{U{a))vj), Vi e Vi, Vj e Vj 

where P is the polynomial deduced from P by changing its coefficients into 
their conjugate. This defines a map, which we call from Dj onto D^. In a 
similar way we have a map tj which is the inverse of Tj. Then, for A e Dj we 

have {Xvi,Vj) = {vi,Ti{\)vj) . 

View Vi as a vector space over Dj. The action 

{X,Vj) ^ Ti{X)Vj 

defines a structure of Dj vector space on Vj. However note that for A G D we 
have ri(A) = A so that Ti{X)vj may be different from Xvj. To avoid confusion 
we shall write, for A G 

Xvi — X*Vi and Ti{X)vj = X*Vj 

As in the first case choose a non zero D— linear form £ on Dj. For f j G and 

Vj G Vj the map A i-^ {X*Vi,Vj) is a D— linear form on Dj, hence there exists 
a unique element S{vi,Vj) G Dj such that, for all A 

(A * Vi, Vj) = i{XS{vi, Vj)) 

The form S* is ©j— bilinear and non degenerate so that we can view Vj as 
the dual space over of the vector space Vi. 

Let {xi,Xj) G EndD(Vi) x EndD(V^). They commute with {ai,aj) if and only 
if they are ©j-linear. The original hermitian form will be preserved, if and 
only if S{xiVi, XjVj) = S{vi, Vj) for all Vi, Vj. This means that Xj is the inverse 
of the transpose of Xi. In this situation we define Gi as the linear group of 
the Dj— vector space Vi. 

We now have to take care of the involutions. Let {^k) be a basis of Vi over 
Dj; let (r/fc) be the dual basis of Vj. Let u be the Dj— linear map from Vj onto 
Vj such that u{^k) — Vk- Then on Gj x Vj x Vj the "local" involution cTj is 

ai{gi,Vi,Vj) = {u~^ ^Qi u,u~^{vj),u{vi)) 
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Next let (ftf.) be a basis of Dj over D and define the dual basis {/Sg)) of Dj by 
£{ar(3s) = ^r,s- Then (a^ * ^k) is a basis of Vi over D and (/^^ * rjt)) is a basis 
of Vj over D. We have (a^ * ^fc,/3s * Vt) = ^r,sSk,t. Using these basis define 
Vi for Vi e Vi and tTJ for vj G Vj-. Then if Xi is a D— linear map from Vi into 
itself we define xl by = Xiiyi) and similarly for T^-. Note that 



Then we define 

7,: y. X y,- ^ X y,- 

by 

^i{Vi,Vj) ^ {-U ^{Vj),-u{Vi)) 

This mapping is D— linear. Indeed if A e D then Xvi — X * Vi whereas 
Xvj — X*Vj so that 

ji{Xvi,Xvj) = {-u'^{X*v~),-u(X* W)) 
= (-X*u'^{v~),-X*u{vi)) 

= X-fi{vi, Vj) 
Furthermore if Vi,Wi G Vi and Vj, G Vj then 

= {u''^{v~),u{wi)) + {u-^{w-),u(vi)) 



= e{Siu-\v-),uiwl)))+iiSiu-^iw-),uivl))) 

Over Dj the duality between and Vj is defined by S and u is self adjoint, 
hence 



{li{vi, Vj),li(m, Wj)) = I (S(wi, Vj) + £ (S{vi, Wj)) 

= {Wl: Vj) + (W, W]) 
= {Vj,Wi) + {Vi,Wj) 

= ((T;i,'t;j),K,Wj)) 

Thus if we extend ji to ^ by letting it act by the identity on the for 
k ^ i,j then ji G G. 
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Let {xi,Vi,Vj) & Gi xVi X Vj. On ViXVj Xi acts as (xj, ^). The transpo- 
sition is taken relative to the duahty over Dj. Let {wi, Wj) & Vi x Vj. 

-fi o {xi, *a;,"^) o-f-^{wi,Wj) = {u~^ *x~'^u{wi),u xlu~^{wj)) 

Thus if we imbed Gi in G by letting it act trivially on \4 for k ^ i,j we get 

Now ai{xi) — ^XiU and a^Xi) = x^"^, therefore 

ad{-ii)xi = (Ti(7(xj) 

The same is true for the action on ViXVj. In other words, for the component 
Vi X Vj the involution a and CTj differ by the action of an element 7i of G. 
Recall that in case 1 they coincide. 

Let M be the centralizer of a. Then {M,V) decomposes as a "product", 
each "factor" being cither of type {Gi, Vj) with Gi a unitary group (case 1) 
or [Gj, Vi X Vj) with Gi a general linear group (case 2). Gluing together the 
involutions we obtain an involution aa on (M, V). We define 7 G G as the 
product of the 7^ ( in case 1 we take 7^ trivial) . Note that 7^ = 1 It is easy to 
check that aa{a) = a. For m G M we have (7o(m) = 7(m)~'^7 = (ad7)((7(m)) 
and for v E V, <7a{v) = 7((T(f )). 

We can now apply Harish- Chandra descent, repeating the arguments of part 
1. By Lemma 5-2 and Lemma 5-3 of this part, there exists an open neigh- 
borhood a; of in the Lie algebra m of M which is closed, invariant under 
the adjoint action of M and such that the exponential map is everywhere 
defined and submersive on ui. Put fl — Exp (a;); it is open and invari- 
ant under the adjoint action of M. Then we may assume that the map 
{g, m, v) H- >• {gmag~^, gv) oiG xVtxV into G xV is everywhere submersive. 
The image of this map is a product U xV with U an open neighborhood of 
a in G invariant under the adjoint action of G. 

By Harish- Chandra submersion principle there exists a map f ^ Ff from 
the space S{G xVtxV) onto S{U x V) such that for any (/? G S{U x V) 

I f{g,m,v)(p{gmag~^,gv)dgdmdv= I Ff{u,v)(p{u,v)dudv 

JgxQxV JuxV 

(the various Haar measures are fixed once for all) . 
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Define 



Hf{m,v) = / f{g,m,v)dg 
Jg 



Then, by Proposition 5-2 of Part 1 tliere is a well defined one to one map 
9 from the space of G— invariant distribution T on U x V into the space of 
M— invariant distributions onQ xV such that, for all / 

{T,Ff) = {e{T),Hf) 

Put 

fi{g, m, v) = f{g, aa{m), 

Then 

Hf^{m,v) = Hf{aa{v),aa{v)) 

Let us compute Fy^. 

/ f{9,CFa{rn),aa{v))(f{gmag~^,gv)dgdmdv = I Ff^{%v)(p{u,v)du dv 
JgxQxV Juxv 

Change m into aa{m) and v into aa{v) 

/ f{g,m,v)(p{gaa{m)ag^'^,gaaiv))dgdmdv = Ff^{u,v)(p{u,v)dudv 

JgxQxV JuxV 

Now 

gaa{m)ag~^ = gaa{am)g~^ = gaa{ma)g~^ = g-fa{ma)-f~^g~^ = a (aig-fy^maaig-f)) 
and 

g(^a{v) = g-f(T{v) = -gjv = a {g^v) = a {(j{g-f)~'^v) 
Therefore, changing g into ^7~^ 

Ff^{u,v) (fi{u,v)dudv 

UxV 

= / f{9,m,v)(p(a {a{g-fy^maa{g-f)) ,a {a{gjy^v))dgdmdv 

JGxflxV ^ ^ 

= / f{gj~^,m,v)ip''{gmag~^,gv)dgdmdv 

JGxCIxV 

— / Ff^(u,v)ip'^{u,v)dudv 

JuxV 
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where 

ip''{u,v) = ip{a{u),a{v)) 

and 

f2{g,m,v) = f{g-f~^,m,v) 

Note that Hf^ = Hf. We just found that Ff^ = Ff^. We also have Hf^ = ifj". 
Therefore if T is an invariant distribution on U x V then 

{e{Tr,Hf) = {e{T),Hf,) = {T,Ff,) = {T,FJ^) = {T%Ff,) = {e{T-),Hf) 

We conclude that OiT") = e{TY^ 

On the other hand we also have a map from ujxV onto VtxV everywhere sub- 
mersive and which commute with the involution a a- Applying again the sub- 
mersion principle we get a one to one map 6' from the space of M— invariant 
distributions onVlxV into the space of M— invariant distributions on a; x y 
and this map 9' also commutes with aa- 

Assume Conjecture 4'. As a; x V is closed inside QxV any distribution on 
u xV extends to a distribution on m x F with support 'm. u xV. Theorem 
D implies that such a distribution , if invariant, is invariant under a. Using 
9' we deduce that any invariant distribution on f2 x K is invariant under aa- 
Using 9 we now get that any G— invariant distribution onU xV is invariant 
under cr. 

Start with an invariant distribution T onGxV and suppose that (t{T) — —T. 
Let g & G and let a be the semi-simple part of g. With notations as above 
we have that the restriction oi T to U x V must be 0. However a belongs 
to the closure of the orbit of g so that for some a; G G we have xgx^^ G U . 
Therefore T is in a neighborhood of {xgx~^} x V but Tis invariant hence 
is in a neighborhood of {g} xV soT must be 0. Any invariant distribution 
is symmetric with respect to a. 

This leaves us with the task of proving Conjecture 4'. The proof will be by 
induction on the dimension n of V . The case n = 1 has been checked. We 
shall assume that the theorem is true for dim V < n and also for the general 
linear case ( Part 1). We take dimF = n. 

The first step is to apply Harish- Chandra descent, this time on the Lie alge- 
bra. This is very similar to what we have done on the group. 
We start with a non central semi-simple element a G 0. It is a D— linear map 
from V into itself such that a + a* = where the adjoint is relative to the 
hermitian form. Let P be the minimal polynomial of a and P = Pi . . . its 
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decomposition into irreducible factors. Put Vi = KerPj(a) so that V — ©V^. 
If M (rcsp. m) is the cogmmutant of a in G (resp. in g) then an element 
m E M (resp. X e m) is diagonal with respect to the above decomposition 
of V. 

Let Dj — D[T]/(Pj). Then Vi has a structure of vector space over Dj. Let r 
be the semi-linear involution on ©[T] such that r(T) = —T and, for A e D, 
r(A) = A. Then r(P) is proportional to P so that for each i the polymomial 
r(Pj) is proportional to one of the Pj. This gives an involutive bijection, 
again called r, of {1,2,..., r} onto itself. As before we have two cases. Case 
1: T{i) — i and Case 2 r(i) = j with i ^ j. With some trivial modifications 
we analyze each case as before. 

In Case 1 we get an hermitian form on Vi relative to some extension Fj of F. 
The group Gi is the corresponding unitary group. In Case 2 the group Gi is 
the general linear group of Vi as a vector space over Dj and Vj is identified to 
the dual of over D,. The group M is isomorphic to the product of the Gj. 
The situation (M, V) is a product of {Gi, Vi) ( case 1) and {Gi, Vi, Vj) (Case 
2). 

For a suitable choice of the basis, with respect to which the conjugations are 
defined, we obtain an involution aa on (m x V), product of local involutions 
CTj. The involution cr on g x y is such that, for some 7 e G 

Ad{-f)a{X,v) ^aa{X,v), X e m, veV 

Because a is not central. Conjecture 4 or 4' is assumed for each component 
Gi- It follows that onmxV any M— invariant distribution is symmetric with 
respect to (Tq. Using Lemma 5-1 and 5-2 of Part 1 and proceeding exactly 
as in the group case we get an M— invariant open and closed neighborhood 
of a in m and, ii U = Ad{G)Q a one to one linear map 6 from the space 
S'{U X y)*^ of G-invariant distributions onUxV into the space 5'(Q x V)'^ 
of M— invariant distributions on Q x V. Furthermore 9{a{T)) = aa{9{T)). 
By induction we know that aa{0{T)) = 9{T) therefore o'(T) = T. 
As in the group case we conclude that if T is an invariant distribution on 
gxV such that cr(T) = — T and if X is an element of with a semi-simple 
part which is not central then T = in a neighborhood of X. 
Let 3 be the center of Q and let M be the cone of nilpotent elements in [g, fl]. 
We have to prove that an invariant distribution T on q x V with support 
contained in (3 x M) xV'is symmetric with respect to a. The involution is 
trivial on 3 and the group G acts trivially on the center so it is enough to 
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consider the case of a distribution with support contained in J\fxV. We will 
now do some reduction on the V side. 

17 . Regular orbits 

We keep our notations. In particular n = dim V. A pair [X, v) & q x V is 

called regular if {v, Xv, . . . , X"'~^v} is a basis of V. 

For any {X,v) e g xV we have {X'^v,v) = {—ly {v^X^'v). If we define 

qi{X,v) ^ {X'v\v) 

then 

{X'v,X^v)^{-iyq(i+^){X,v) 

When no confusion may arise we write qi for qi{X, v). Then define the nxn 
matrix A{X, v) ( or simply A) by 

A{X,v) = {ai^j) with ai^j = {X'-^v\X^-h) 

Let D{X, v) be the determinant of A{X, v). 

Proposition 17.1 {X,v) is regular if and only if D{X,v) ^ 0. The set of 
regular elements is a non empty Zariski open subset of QxV. 

If (X, v) is regular then A{X, v) is the matrix of (., .) in the basis v, Xv, . . . X"^^v 
and as the hermitian form is non degenerate, the matrix is non singular. 
Conversely if there exists a non trivial linear relation among the vectors 
V, Xv, . . . X"'~^v then the same relation will be true for the rows of A which 
is thus singular. 

The second assertion simply means that D ^ 0. Fix a basis ei, . . . e„ oi V 
with coordinates Zi, Z2, ■ ■ ■ Zn, such that 

n 

{V,v) = ^\jZjZ] 

1 

Here the are non zero elements of F. Define X by 

Xei = 62, Xe2 = -(A1/A2) €1 + 63,..., 
Xsn-i — — (A„_2/ A„_i) e„_2 + e„, Xcn = — (An-i/ A„) e„_i 
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Then X G and (X, Ci) is regular. 
For X e define the Dj{X) by 

n-l 

Det(rid - X) = T'^ - ^ Dj{X)T^ 



Like the the Dj are invariant polynomial functions, with values in D. Note 
that the qj and Dj are not algebraically independent. 

Proposition 17.2 Two regular elements are conjugate under G if and only 
if they give the same values to the invariants qj and Dj . 

The necessity is clear. Conversely let (X, v) and (F, w) be two regular ele- 
ments such that 

q,iX,v) = qj{Y,w), D,{X,v) = D,{Y,w) j = l,...n-l 

In particular 

A{X,v) = A{Y,w) 

so the linear map g from V toV defined by g{XPv) = Y^w for p = 0, ... n — 1 
belongs to G. We claim that gXg~^ — Y. It is enough to check that 

g-^YgXPy = XP+\, p = 0, . . . n - 1 

Now, if p < n — 2 

g-^YgXPy = g'^Y^+^w = X^+^t; 

For p — n — 1 

n— 1 n—1 

g-^YgX^'-'^v = g'^Y^'w = c/"^ ^ Dj{Y)Y^w = ^ Dj{X)Xh = X^t; 



We define the regular orbits as the orbits of regular elements. Each such 
orbit is defined by the values of the qj and the Dj. These values must be such 
that £), which is a polynomial in the qj should not take the value and also 
the relations between the qj and Dj must be satisfied by the chosen values. 
Each such orbit is (Zariski) closed (the invariants are constant on the closure 
of any orbit !). If (X, v) is regular and if G G is such that 5'(X, v) = (X, v) 
which means that gv — v and gXg~^ — X, then gX^v — {gXg~^Ygv — X^v 
for all p. By definition of regular {v, Xv, . . . , X^~^v} is a basis of V so we 
conclude that g — Id: the isotropy subgroup of a regular element is trivial. 
In fact 
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Theorem 17.1 An orbit is regular if and only if it is closed and if the cen- 
tralizer in G of an element of the orbit is trivial. . 



It is enough to prove that if the space E generated by the X^v for p — 
0, . . .n — 1 is a proper subspace of V then either (X, f) has a non trivial 
isotropy subgroup in G or the orbit of (X, v) is not closed (or both . . . ). 
Suppose first that the restriction of (., .) to £^ is non degenerate and let F be 
the orthogonal of E. Then V — E ® F, an orthogonal decomposition. We 
know that X{E) C E, hence X{F) C F. Relative to the above decomposi- 
tion X may be writen as 



where X3 belongs to the Lie algebra of the unitary group of the restriction of 
(., .) to F and commutes with X2. We may choose X3 7^ Then Y E Q and 
[y, X] = and also Yv = 0. The orbit of {X, v) is not of maximal dimension, 
hence is not regular. 

In general the restriction of (., .) to E is degenerate. Let N be its kernel and 
choose a subspace Ei of E such that E = N ® Ei. Note that X{N) C N 
and X{Ei) C N ® Ei. Let E2 be the orthogonal complement of E^. Then 
V = Ei(B E2 and N G E2. Fix a decomposition 



such that the restriction of (.,.) to E2 is anisotropic, the orthogonal com- 
plement of E2 in E2 is E2 ® -B^, the subspaces E2 and E2 are maximal 
totally isotropic subspaces of E2, the form (., .) is a non degenerate (hermi- 
tian) duality between E2 and E2 and finally N C i?^. Let M C E2 be a 
subspace such that E2 = N (B M. Viewing E2 as the (anti)dual of E2 we 
get a corresponding decomposition E2 = N~ © M^. Then 




Put 




E2^E^®E+® E- 



V ^Ei®{N®M)®{N~ ® M~) © E° 



For t e F* let at be the endomorphism of V defined by 



{at)\N = ^Id, (at)|jv- = t 
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and at — Id on the other components. One checks easily that at G G. 
U V — Vq + Vi with vq & N and Vi e Ei then at{v) — tvo + Vi so that 

lim atV — Vi. 

Let u & V and suppose that Xu has a non zero component in A^~; then for 
some n ^ N we have {Xu, n) ^ hence {u, Xn) 7^ 0. However Xn G N 
hence this imphes that u has a non zero component in . 
Let us compute atXat^. First if n~ e A?"" then atXat^n~ — tatXn~ so that, 
when t goes to 0, the hmit is the N~ component of Xn". Next ii n E N 
then Xn G A?" and a^Xa^^n = Xn. Finally if m G K has a zero component 
relative to NQ)N~ then a^^'^-u = u and we saw that Xu has a zero component 
relative to N~ . It follows that the limit of atXat^u when t goes to is the 
projection of Xu onto the subspace 

E^®M®M-®El 

We have checked that atXat^ has a hmit Xi. The point (Xi,^;i) belongs to 
the closure of the orbit of (X, v). Relatively to the decomposition E — N®Ei, 
the restriction of X to £^ may be written in matrix form 

= (0 z) 
and the restriction of Xi to E is then 

™i^^(o z) 

It follows that Xfvi is the projection onto Ei of X^v. Thus the subspace 
generated by Xfvi, p = 0, . . .n — 1 is Ei. If X 7^ (0), (which we tacitly 
assumed) then {X,v) and {Xi,vi) are not conjugate. Indeed if, for some 
g & G we have gXg~^ = Xi and gv = Vi then, for all p we get gX^v — Xfvi 
so that g{E) = Ei which is impossible because dim(£') > dim(£'i). 
We have a more precise result. 

Let p be the dimension of E. Let q be the largest integer such that the vectors 
V, Xv, . . . , X'^~^v are linearly independent. Then X'^v is a linear combination 

9-1 

X'^v = ^ arX^'v 
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which imphes that the subspace generated by the X'^v, r — 0, ... ,q — 1 is 

stable by X hence must be equal to E. We have p = q. 

If the restriction of (., .) to E is non degenerate, let F be the orthogonal of 
E. Then F is stable by X. Relatively to the decomposition V = E (B F 



Note that Xe (resp. Xp ) belongs to the Lie algebra Qe (resp. Qp) of the 
unitary group Ge E (resp. Gp F). 

Theorem 17.2 For the orbit of {X, v) to be closed it is necessary for the 
restriction of {., .) to the subspace E to be non degenerate. If this is true then 
the orbit is closed if and only if Xp is a semi-simple element of Qp. 

The first assertion has just been proved; let us prove the second one. Suppose 

that the orbit is closed and let G 0_f be an element of the closure of the 
orbit KdGp{Xp). There exists a sequence Un of elements of such that 
gnXpg~^ converges to Yp. Define 





Then gn & G and gnV — v so that gni^, v) converges to 




The orbit is closed so there exists g G G such that gv 



V and 




For all q we have 




so that g is the identity on E. Thus we may write 
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So Yp = u{Xf). The orbit of Xp is closed which is equivalent to the semi- 
simplicity of Xp. 

Conversely assume that Xp is semi simple and let {Y,w) = limgn{{X,v) be 
some element of the closure of the orbit of {X, v). We know that limgn{X'^v) ~ 
y^w. In particular the linear relations among the X'^v remains valid for 
the y^w. The subspace E' generated by the V^w is thus generated by 
w,Yw, . . . ,Y^~^w. We claim that dimE' is exactly p. Indeed wc have 



{X''v,X'v) = {Y''w,Y'w). Now v, Xv, . . . X^-^v is a basis of E and (., .) 



is also non singular which implies that the vectors w, Yw, . . . , Y^^^w are 
linearly independent and that the restriction of (., .) to £" is non degenerate. 
There exists u G G such that u{X'^v) = Y'^w for g = 0, 1, . . .p — 1. To 
prove that {Y, w) belongs to the orbit of {X, v) it is enough to prove that 
u~^{{Y, w)) belongs to this orbit. In other words we may assume that, for all 
e E E we have limg'„(e) = e; in particular w = v. 

Consider the decomposition V = E (B E. The hermitian form (., .) defines a 
semi-linear map s from V to its dual. Put 



For any linear map from V to V let /* = {^f)s be the adjoint map; we 
use a similar notation for E and E. 
Write gn in matrix form 



restricted to E is non degenerate so that the matrix 




is non singular. Hence the matrix 






«n«n + 7n7n = 1 
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The condition limg'„(e) — e ior e & E means that 

hm7„ = 0, hma^ = 1 

In particular q;„ is invertible for n large enough and from the third equation 
above we get 



and 
Thus 

which we rewrite as 
with 



(5:)-^((5„)-i = i+£„ 



Note that En = and has limit 0. Consider the map f ^ f*f from the 
space of self adjoint maps from F to F into itself. The differential at the 
origin is h ^ h* + h, a bijection. Hence we have around the identity a local 
diffeomorphism. Forgetting about a finite number of values of n, we see that 
there exists a sequence /„ of maps from F to F, converging to 1 and such 
that f*fn = I + En- Going back to Sn we have 



or 



ifJuTifJu) = 1 

Then Un = fn^n & Gp and 

fn 

Now 

is explicit ed in matrix form as 

( Oin I3n\ ( Xe W = ( "n^S«n + l^nXp^l OIuXeII + f^nXp^l 
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The limit of this matrix is 



Y 



Xe 
Yf 




The left side belongs to the orbit GpXp so its limit Yp belongs to the closure 
of this orbit. However Xp is supposed to be semi simple so the orbit is closed 
and we get that Yp — uXpu~^ for some u e Gp. Finally put 



Finally let us note that it follows from Theorem 17-1 that a regular orbit 
always carries a G— invariant measure. Also a regular orbit is stable by a 
(Proposition 17-2) so that this invariant measure is stable by a. it follows 
that any invariant distribution on the regular set is symmetric with respect 
to a. 

18 . Reduction to the singular set 

Let X e g and v E V. Consider the sequence of vectors (X^v) . As X" is a 
linear combination of the X^ for j < n, for s ^ n, X^v is a linear combination 

of the X^v for j < n. Let r be an integer, 1 < r < n and choose a class 7 
of hermitian forms, non degenerate and of rank r. We shall denote by S(7) 
the set of all (X, v) such that the subspace 




Then g E G and gv = v and gXg 



1 



Y. 



r-l 
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is of dimension r and such that the restriction of (., .) to this subspace belongs 
to the class 7. Note that , for r = n, if (., .) does not belong to the class 7, 
then S(7) = and if (., .) belongs to 7 then ^(7) is the subset of regular 
elements. 

Lemma 18.1 Each 2(7) is a, possibly empty, open subset. The union of 
all 2(7) is the set of all {X,v) such that En{X,v) is not a totally isotropic 

subspace 

Indeed,the linear independance of the X^v. j = 0, . . . , r — 1 and the non 
degeneracy of the restriction of (., .) to Er{X,v) are equivalent to the non 
vanishing of the determinant of the matrix 

Ar{X, v) = {{X'-\, X^-\)) , i,j^O,...,r-l 

Once r is fixed, there is only a finite number of classes 7 characterized by the 
discrimant, the determinant of the above matrix, modulo the squares (in the 
base field F) and the Hasse symbol in the case of quadratic forms (© = F). 
To prove the second assertion let qj = {X^v, v). If Qj = for j = 0, . . . ,n — 1 
then all the qj are and the restriction of (., .) to En{X,v) is 0. Otherwise 
let r be the smallest integer such that Qr-i 7^ The matrix 



Ar{X,v) 



( 






V?r-1 








* 








-?r-l * 
* 



is non singular. Hence {X.,v) belongs to some S(7). 

Assume that there exists a subspace E of 1/, of dimension r, such that the 
restriction of (., .) to E belongs to a given class 7. Let ^(7) be the closed 
subset of 2(7) defined by E^iX^^v) = E. By Witt's theorem G'E{'~f) = ^(7). 
Suppose that {X,v) G ^(7), that g E G and that g{X,v) G 2(7). Put 
w — gv and Y — gXg~^. Then {w,Yw, . . .Y'^~^w} is a basis of E. But 
giX^v) = Y^w so g{E) = E. This implies that g{E{^)) = 2(7). Let H be 
the stabilizer of ^(7) in G or what amounts to the same the stabilizer of E 
in G. For any subspace F of y call Gp the unitary group of the restriction of 
(., .) to F and Qf its Lie algebra. Then H is isomorphic to x Ge^ where 
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is the orthogonal complement to E. We are in a situation to apply the 

results of the Appendix . 

Use the decomposition V = E-^ (B E to write X in matrix form 

X = ( "^^'^ "^^'^ 

V 2^2.1 2^2,2 

Also the hermitian form (., .) defines a semi linear map from V to V* which 
we write as 

(s^ 
I s 



Then X e g is equivalent to 



*X2,2S + SX2,2 = 



5X2,1+ Xi^2S = 

If {X,v) e 2(7), for j < r — 2 we have X^v — ^2^2'^ and x-i^2X^v — The 
linear map Xi,2 from E to E-^ is of rank at most 1. More precisely there is a 
unique w & E such that 

r-1 

XjX^V, w) — Xr-l 



and if we define u G E-^ hj u = xi 2X^~^v then, for e & E, we have a;i,2(e) = 
{e,w)u. Thus starting from {X,v) G ^(7) we obtain, on one hand (a;i,i,-u) 
which belongs to x E^, with no particular condition and on the other 
hand {x2,2,v) which is a regular element of Qe x E. 

Conversely if we start with two such elements {xi^i, u) and a regular (3:2,2, v) 
then we define w using v and 0:2,2 and then 2:1,2 using w and u which allows 
us to recover X. Thus if we call (g^; x i?)reg the open set of regular elements 
in Qe X E then we have an homeomorphism 

(Qe^ X E^) X (Qe X £;)reg ^ 2(7) 

Write an element h oi H as 



h 



Hei- 
He 
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Then, for (X, v) e 2(7) we get hv — hsv and 



hXh-^ = i 



For e e ii^ 



hE±Xi^2hE\e) = {hE\e))hE±{u),w) = {e, hEiw))hE±{u) 
But hE{w) satisfies 

{hExi2hE\hEv),hEiw)) = {xi^2v)\w) j = 0, 



so that, under the action of h, the vector w is replaced by hE{w) and we get 
that the action of H on 2(7) is the product of the actions of the two smaller 
unitary groups. 

We will now draw several consequences of the above remarks. First the 
G— orbits in ^(7) are in one to one correspondance with the i7— orbits in 
5(7) which themselves are obtained by taking the product of a regular orbit 
of Ge in Qe X E by an orbit of Ge^ in t)gE± x E-^. Let {X,v) G 2(7). 

Proposition 18.1 {X,v) is regular if and only if {xi^i,u) is regular 

Indeed by induction one checks that there exists constants Cqj such that 



We know that (X, v) is regular if and only if X^v, j = 0, . . . , n — 1 is a basis 
of V. In all cases X^v, j = 0, . . . , r — 1 is a basis of E thus (X, v) is regular 
if and only if xl^iU, g = 0, ...,n — 1 — risa basis of E-^ which means that 
(xi^i, u) is regular. 

Proposition 18.2 Let {X,v) G 5(7). The orbit G{X,v) is fixed by a if 
and only if the orbit Ge^{xi^i,u) is fixed. If any Ge^— invariant distribution 
on E-^ is symmetric with respect to the involution Ue^ then any invariant 
distribution on ^(7) is symmetric with respect to the involution a. 

To define cr, we may choose a basis ei, . . . , oiV over B relative to which 
the hermitian form is diagonalized: 




5-1 
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Using the coordinates relative to this basis we define v^X^. . . and a{X^ v) — 
(— X, —v). A different choice of basis will simply replace a hy g oa for some 
g E G so that the above Proposition is independent of this choice. We choose 
a basis adapted to the decomposition V = E Q) E-^. 

With the same notations as above, if we replace (X.v) by a{X, v) = {—X, —v) 
then u is changed into {—lY~^^u and Xu into — Xi^,so that {xi^i, u) is replaced 
by (— {—ly^^u) = cr{xij. (— !)'«). However in E-^ the map —Id belongs 
to the center of the unitary group so 

(7(xi,i, {-lyu) = (7 o (-Id)''(a;i,i, u) 

On the other hand (^2,2,1') is replaced by (— ^2,2, — = o'{x2,2,v). As any 
regular orbit is stable we get the first assert ion. Also for invariant distributions 
our map commutes with a which implies the second assertion because on the 
regular set an y distributionn is symmetric. 

Going back to our general situation, as we proceed by induction we may 
assume the theorem for E-^. Therefore it remains to consider the case of 
invariant distributions with support contained into the set of all {X, v) such 
that {X^v, v) = for all j. On the other hand we also know that it is enough 
to consider distributions with support contianed into the set of all {X, v) 
with X nilpotent. 

We shall say that {X,v) is singular if X is nilpotent and if, for all j, 

{BX^v,v) = 0. Let E be the singular set. We have to prove that any 
invariant distribution with support contained into the singular set is stable 
by a. 

19 . The singular set 

We keep our notations. An invariant distribution T on y is called symmetric 
(resp. skew symmetric) if (j{T) — T (resp. cr{T) — —T). An element 

(X, f ) G g X 1/ belongs to E if and only if the subspace E{X^ v) generated 
by the vectors X^v is totally isotropic and if X is nilpotent. Let E^ be the 
subset of E defined by dim E{X,v) = r; then 

e = |Je, 


where Tq is the largest possible dimension for a totally isotropic subspace of 
V. 
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The subset S^. is defined by the following conditions: the invariants Dj{X) 
and qj{X, v) are all 0, the vectors t;, Xv^ .... X^^^v are linearly independant 
and X^v = 0. Indeed as E{X,v) is stable under X which is nilpotent the 
restriction of X^ to E{X,v) is 0. In particular is locally closed, hence is 
an Ictd space and we may introduce the spaces S{T,r) and (S'(Er). Also Ej. 
is stable by a. We will show in a moment that 7^ for r < Tq. 
We claim that in order to prove our theorem it is enough to prove that, 
for each r, an element of 5'(Er) which is invariant under G is symmetric 
. Indeed, let T be an invariant distribution supported on E and suppose 
that it is skew symmetric. The restriction of T to the open subspace E^o 
of E is invariant and skew symmetric. If we have proved that an invariant 
distribution on E^g is always symmetric we conclude that this restriction is 
0. Next we restrict to E^o-i an open subspace of the complement of E^g in E 
and so on, obtaining finally that T = 0. As the involution commutes with the 
action of G, any invariant distribution is the sum of an invariant symmetric 
distribution and a skew symmetric one, and this is enough to conclude. 
From now on we fix r < tq, r 7^ .We shall deal later with the case 
r = 0. Two totally isotropic subspaces of dimension r are conjugate un- 
der G. Fix one of them E. Up to conjugation by G we may assume that 
{v, Xv, . . . , X'^~^v} is a basis of E. Choose a second totally isotropic sub- 
space E*, of dimension r such that the sum E + E* is direct and that (., .) 
is a non degenerate (semi) duality between E and E*. Let F — {E ® E*)-^. 
We write V as 

V = E®F®E* 

Fix a basis {cq, . . . , e,._i} of E and let {cq, . . . , e*._i} be the dual basis of 
E* so that (ei,e*) = 6ij. Choose a basis (uj) of F such that {uj,Uk) G F. 
Then, relative to these choices, the hermitian form (.,.) has a matrix sy 
which is equal to its adjoint ( transpose of the conjugate) and in fact has 
its coefficients in F .In matrix notations the hermitian form is then v*Svv' — 
{v',v), the unitary group is defined by g*svg = sy and the Lie algebra by 
X*sv + syX = Relatively to the decomposition V = E (B F Q) E* in matrix 
form 









Id 





s 





Id 
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If /i e GL{E) then 

/h \ 
g = i Up 

\o (/i*)-v 

belongs to G. Up to conjugation by G we may assume that X^v = e^-i-j 
for j = 0, . . . , r — 1. . Let be the set of aU (X, v) satisfying the above 
conditions. Note that it is a closed subspace of and we know that GS^ = 

. 

If (X, v) e then v — e^-i and, making explicit the condition X e we 
see that 





f n-r 


-b*s 


a 







Y 


b 




^0 





-< 



where is defined by nj.{ej) = ej_i for j = 1, . . . , r — 1 and nr(eo) = 0. Also 
a + a* = and 6 is arbitrary. Finally Y E t)gp : Y*s + sY = and X is 
nilpotent if and only if Y is nilpotent. 

Let {X,v) e Er and let gi e G be such that g{X,v) e S^. Put g{X,v) — 
{X', v') so that V — v' — Cr-i and X' = gXg~^. Then g'(er_i) = e^-i and 

5i(er-i-j) = (5rX-')(e,._i) = {X'^g){er-i) = X'-'(er-i) = e^-i-j 

Thus is the identity on E. Conversely if G G is the identity on E then 
g'E.r — S^. Let H be the subgroup of g & G which are the identity on E. 
U h E H, then, expliciting the condition h E G we get 

(1 —x*sy z 

y X 

1 

where x,y,z are such that 

y*sy = s 
z + z* + x*sx — 

The reductive part of H is isomorphic to Gp, the unitary group of the restric- 
tion of the form to F. We can now use the results of the Appendix. There 
is a one to one correspondence 6 : S ^ T from the space of iJ— invariant 
distributions on onto the space of G— invariant distributions on such 
that 

(T,/)= / {S^J{gO)d9 
Jg/h 
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Now consider the involution {X,v) ^ {—X,—v) ( we use the above basis 
of V). The strata E,. is stable but not S,,. To correct the situation, define 
te ■■ E ^ E hy TE{er-i-j) = {-ly^^er-i-j and r e G by 






If {X,v) e Er then t{—X, —v) belongs to S^.. Indeed v — e^-i — v — —r{v) 
and, using the above notations for X we get 

r(-X)r-^ = 

use 5'(S,)^ 

{e{S)J{-X,-v)) = [ {S,f{g{-X,-v)))dg 

Jg/h 

{SJ {gr-\r{-X)T-\v)))dg 



G/H 

= / {S,f{g{T{-X)T-\v)))dg 

JG/H 

For the last equality note that r normalizes thus operates on G/H by 
right multiplication and being an involution it leaves the measure on G/H 
invariant. The distribution T = 9{S) is symmetric if and only if the distri- 
bution S is symmetric relative to the involution {X,v) i— > (— rXr~^,f) of 

Our next step is to use the invariance under the unipotent radical U of H. 
First note that v does not play any role so we may as well consider Sj. as a 
subset of 0.1f 



X = 
then 

uXu'^ 




a \ 


/ 


h , 


and ti = j 1 


-KJ 




( rir 


-{b'Ys a' 


Ho 


Y b' 




-n; 
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with 



h' = h-{Yx + xn;) 

a' — a — {urZ + z n*) — rir x*sx + x*sYx + h*sx — x*sb 

Also in the sequel the fact that Y is nilpotent will play no role whatsoever. 
So we will just forget about this assumption and take for Y any element of 
t)gp. We study the map L : x i— > Yx + xn* from Hom(£'*, F) into itself. Let 
Cj — x{e*) and 7^ = L{x){e*). Then 



In particular L{x) = if and only if cj = {~l)^Y^Co with Y''co = 0. Hence 
the kernel of L is isomorphic to the kernel of Y^' . Next we look for the x 
such that 7o = 7i = ■ ■ ■ = 7r-2 = 0. The conditions are Cj = {—lyY^co for 
j — 1, . . .r — 1 and then ^^-i — {—^Y'^Y^ Cq- In other words the intersection 
of the image of L with the subspacc 70 = . . . = 7^-2 = is the subspace 
7o = . . . = 7r-2 = 0,7r_i G ImFr- Its dimension is equal to the rank of 
y, hence if Fq C F is any subspace supplementary to this image then the 
subspace Fq : 70 = . . . = 7r-2 = 0, 7r._i G Fq has the same dimension as the 
kernel of L and intersects trivially the image of L hence is a supplementary 
subspace of the image of L. It follows that if X G then up to conjugation 
we can assume that h G Fq. At least for the moment we will just remember 
that, up to conjugation we may assume that h{e*) = for j < r — 2 (in 
matrix terms, only the last column of b is non zero), let F be the space of 
all such b. 

Let us take x — 0; only a is affected and now we have to study the map 

N : z ^ UrZ + zn* with z + z* = . We use matrix notations with z = (ztj) 
and N{z) = (Oj)- Recall that < i, j < r — 1; we make the convention that 
Zr,j = Zi^r = 0. Then 



Here we deal with the structure of vector space over the base field F. The 
kernel of N is the subspace of anti-hemitian matrices z such that 



7i = Ycj + Cj+i, i = 0, . 



r - 2, 7r_i = Ycr-i 





{—lyzi+jfi if i + j <r and i + j odd 
if i+j^r or i + j even 



Its dimension is r/2 if r is even and (r — l)/2 if r is odd. 
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A straightforward computation shows that the elements of the image of 
such that Qj = for j < r — 2 and i < r — 2 (only the last column and the 
last row are non zero) are the anti-hermitian matrices of size r such that the 
last row is 

( q;3 q;2 q;i 0) 

with all the ctj G F. Then denote by Eq the subspace of E of all vectors 

... + ... + Ps^r-e + /52er-4 + Pier-2 

with again /?i G F and let Eq be the space of all antihermitian matrices such 
that only the last column and last row are non zero and such that the last 
column belongs to Eq. Then the dimension of Eq is equal to the dimension 
of the kernel of A^, the intersection of Eq with the image of A^ is reduced to 
(0), so that Eq is a supplementary subspace of the image of A^. 
Let 6^ C be the set of all X eEj. such that Y e Qf ,a e Eq and b E F. 
we have just checked that UQr — '^r- We cannot use the Appendix but 
Harish-Chandra 's submersion principle does apply. Indeed both and 
are vector spaces, hence analytic manifolds ans we have a surjective map 

TT : [/ X ©r ^ 5r 

given by 7r{u, ^) = u^u~^. To compute the differential of tt we evaluate 

7r(uEx.-p(tA),^ + r]), Aeu^Ue{U), rj e Or 
The differential is 



(Ar;)^AdH(r/+[Ae]) 



If 



then 



/o 


-(3*s 


a 




















/O * 













I 





/ Ur —b*s a 
b 



Y 
\0 



[a n* + Ura) + {b*s(3 - /3*sb) 
-{Yp + P*nr) 




By definition of ©^ this formula shows that = ©^ + Ad(^)u and the map 
is everywhere submersive. 
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Thus there exists a surjective map / i— > Ff oi S{U x O^) onto <S(Sj.) such 
that, for any (p e 5(5^) 



/ 

Juxe, 



Here dii, and dX are Haar measures fixed, a priori, once for aU. By 
transposition we obtain a one to one map 9 : S ^ T from S'i^r) into 
S'iU X e^) such that (T, /) = {S,Ff). 

Let g E U; we have (/(^fM) = and d(Ad5'(X)) = dX. Put f^{u,^) = 
Ff4XMX)dX = [ f{g-\,Ov{n{u,0)dud^ 

JUxOr 

= / f{u,0^{Adg7riu,0)dud^ 

JUxQr 

Ff{X)ip{Adg X)dX 



= J^{FfnxMx)dx 



This imphes that {FfY = Ffg and that S is [/—invariant if and only if 
T = ^(5*) is [/—invariant. If this is the case then we may decompose T as 
T = duR where is a distribution on 0^. 

Now consider the action of the reductive part of H that is to say oi Gp- If 
y & Gp, imbedded in if, it acts on simply by changing b into yb and on Y 
by the adjoint action. In particular is fixed. Put ^f{u,^) — f{u,Ady{^)) 
and fy{u,C) = f{y~^uy,C). 



L 



[ Fyf{XMX)dX = [ f{u,Ady{0)^{TT{u,0)dud^ 

f(u,0^{7r(u,Ady-\0))dud^ 

Ux0r 

f{u,Ov{Ad{uy-')^) dudi 

UxQr 

f fy{u,0^{Ad{y-')7r{u,^))dud^ 

JUxQr 

I Ff^{X)^{Ady-\X))dX 

J St- 
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L 



Ff^{My{X))ip{X)dX 



This implies that 



Fyf{X) = Ff^ (Ady(X)) 



Suppose that the distribution S is invariant under H. Then it is invariant 
under U and y. As before put T — 9{S) — duR. Then, by the biinvariance 
of du we have 



The conclusion is that T is invariant under y and 5* h- > i? is a one to one 
map from the space of i7— invariant distributions on 5^ into the space of 
Gi?— invariant distributions on O^. 

Let us look at the involution X ^ t{—X) on S^. Take X e O^- Then a e F 

and TEdT^ = —a and —br^ = b. As r leaves the Y component invariant we 
get that, on 0^ the involution is given hj Y —Y, b i-^ —b, a ^ a. 



Put '^fiu^i) = /(m, Adr(— ^)) and /r(M, = /(''" This makes sense 



because adr leaves U invariant and note that du is also invariant by adr. 



{T,f) = {T,fy) = {S,FfJ 



By the invariance of S under y 



{S, Ff^iX)) = {S, Ff^ {Ady{X))) = {S, Fyf) 





-1 



)C) dud^ 




This implies that 



Frf{X) = Ff^ (Adr(-X)) 
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or, replacing / by fr 

Fr^f^^iX) = Ff {AdT{-X)) 
If 5" is iJ— invariant and 9{S) —T — du®R then 

(Adr(-X))) = = (T,^(/0) 

Assume, for a moment that T is symmetric : {T,'^ f) — (T, /) , then 

(T,-(/0) = (T, A) = {du ® R, f{r-'ur, 0) = (T, /) = (5, Fj) 
and is symmetric. 

Therefore we must prove that T = du® R\s symmetric. First suppose that 
r > so that F is of dimension strictly positive and strictly smaller than 
the dimension of V. identify Qr with {qf ® F) ® Eq. Let a e S{U) and 
13 e S{Eo). On ® F the distribution 

'tp{Y,b) ^ {T,apij) ^ I a{u)du{R,ip{Y,b)p{a)) 

Ju 

is invariant hence symmetric by the induction hypothesis and this is enough 
to prove that T is symmetric. 

Finally if r = then v = so that Eq is essentially Q and we have to prove 
that any invariant distribution on Q which is supported on the nilpotent set 
is symmetric. We know that the invariant measures carried by the nilpotent 
orbits are a basis of this space of distributions so it is enough to prove that 
if X is nilpotent then X and —X are conjugate. 

Proposition 1-2, Chapter 4 of [6] asserts that there exists an F— hnear map g 
from VtoV such that gXg—^ — —X and {g{v), g{v')) — {v', v); in particular 
g{Xv) = Xg{v). As usual let us define the conjugation with some basis of V 
such that the hermitian form has a matrix with coefficients in F and put 
6{v) = V. Then 6g is D— linear, unitary and {6g)X{6g)~^ = —X. 
This completes the proof but we assumed the case of the general linear 
group. . . 
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Appendix: A Probenius type descent 



If X is a HausdorfF totally disconnected locally compact topological space 
(Ictd space in short) we denote by S{X) the vector space of locally constant 
applications with compact support of X into the field of complex numbers C 
. The dual space S'{X) of S{X) is the space of distributions on X. Once for 
all we assume that all the Ictd spaces we introduce are countable at infinity. 
If an Ictd topological group G acts continuously on a Ictd space X then it 
acts on S{X) by 

and on distributions by 

The space of invariant distributions is denoted by S'{X)'^. Let G be a Ictd 
topological group and H a closed subgroup. Suppose that H acts continu- 
ously on a Ictd space X. Let H acts on G x X by 

{h, {g,x)) ^ {gh~^,hx) 

and let Y be the quotient space. The equivalence relation is open and its 
graph is closed, hence Y is HausdorfF and a Ictd space. Let tt be the projection 
map from G x X onto Y. The group G acts on G x X and on Y on the left: 
9(9', x) = (gg',x) and g(7r(g',x)) = n{gg',x) 

Let e be the neutral element of G and consider the subspace {e} x X. We 
have 

7r-^(7r({e} x X)) = H x X 

a closed subspace. Hence vr({e} x X is closed in Y and as vr is open it follows 
that X is homeomorphic to 7r({e} x X). We shall identitfy X with this 
image. So X is a closed subspace of Y, we have GX — Y and if, ior g e G 
there exists x e X such that gX e X then g E H. Finally HX — X. Note 

however that n{h{{e,x)) = 7r{{h,x) = vr((e, h~^x)) 

Let Ah be the module function of H and the module function of G Let 
5a(G) be the space of functions ip defined on G, locally constant, such that 

^{gh) = ^^^{g), geG,heH 
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and with support compact modulo H. Then there exist a positive hnear form 
^ on this space, invariant by right translations by elements of G; it is unique 
up to multiplication by a strictly positive number. We use the notation 

i (p{g)dfi{g) 

J G/H 

If dig is a left Haar measure on G and drh a right Haar measure on H we 
may normalize in such a way that 

/ f{g)-dg^ I I f{gh)AG{h)drhdpi{g) 
Jg Jg/h Jh 

Let S\X)^ be the space of distributions S on X such that 

Let S G S'{X)^ and define a distribution T = 6{S) on Y by 

{T,f)^<f {S,f{gx))dii{g) 

JG/H 

To check that this makes sense consider the continuous map {g-ix) i— > gH of 
G X X onto G/H . It defines a continuous map v from Y to G/H . liy ^Y 
and ii g E G and x E X are such that y = gx then z/(i/) = (jfiJ. 
Let U be the support of /; it is compact and open. Then gx E U implies 
that gH e i^{U) so that the support of {S,f{gx)) is compact modulo H. 
Furthermore / is fixed by some open compact subgroup of G and the same 
is true for {S, f{gx)); we can apply /i. The distribution T is invariant under 
G. 

Proposition 19.1 The map 6 is bijective. Furthermore for any f e S{Y) 
there exists (f e S{X) such that, for any S e S\X)^ 

{s,cp) = {e{s)j) 

For F e S{G X X) put 

f{gx) = / F{gh'\hx)drh 
Jh 
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Then / is well defined and / e S(Y). The map F i— > / is onto. Also note 
that the support of / is contained in the projection of the support of F. The 
transpose map T i— ?7 is one to one linear from S'{Y) into S'{G x X) and 
its image is exactly the subspace of distributions on G x X which under the 
action of H satisfy 



The group G operates on Y and also on G x X, by left multiplication on the 
first variable {{g', {g,x)) ^ {g'g,x). Then T i— > is a bijection from S'{Y)'~' 
onto the space S'{G x X)'^'^ of distributions on G x X invariants by G and 
having the above invariance property with respect to H. 
Such a distribution U is uniquely written as a tensor product d^g <S> S where 



{U,F{gh-\hx)) = AHih){U,F{g,x)) 



S e S'{X)^. Then e{S) = T. 

Indeed, for / e S{Y) choose a "hft" F. Then 



{0(3), f) = / {S,f{gx))dM 



Jg/h 




G/H JH 




= {dig®S,F) 
- {U,F) 
= (TJ) 



Finally for / as above, choose F and define 




Then, for all S we have {S, ip) = {9{S), /). 



For our peace of mind here is a detailed proof. 
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Lemma 19.1 Let Y be a locally compact totally disconnected space which is 
countable at infinity. If 

Y = [jU, 

iei 

is an open covering of Y, then there exists a finer covering 

locally finite and such that each Vj is open and compact. Furthermore we 
may assume J to be countable. 

Let Yn an increasing sequence of open compact subsets of Y such that Y — 
DYji- Let Wn — — K^-i, n ^ 1 and Wq — Kq. Then each Wn is open and 
compact. For each y G Wn choose i e / such that x E Ui and let be an 
open and compact neighbourhood of x contained in WnUi. Wc can cover Wn 
by a finite number of such V^. Letting n vary we get a covering of Y with 
the required properties. 

With the notations of the lemma, let gj be the characteristic function of 
Vj. Because the covering is locally finite, the sum 'Y^gj is well defined and 
strictly positive. Put Kj = gj/Yli9r- Then each Kj G S(Y), the support of 
Kj is Vj and ^f^j = 1 . We shall call the familly of Kj the partition of unity 

associated to the covering Vj. 

Lemma 19.2 Let H be a Ictd group, and Z a Ictd space countable at infinity. 
Suppose that H acts continuously and properly on Z , on the left. There exists 
a locally constant function u defined on Z , strictly positive whose support has 
a compact intersection with the inverse image of any compact subset of H\Z 
and such that 

u{hz)drh — 1 

On if \ Z we put the quotient topology. The equivalence relation is open, 
hence the projection map tt from Z to H\Z is open. Because the action is 
proper, the quotient is Hausdorff. Hence this quotient is also a Ictd, countable 
at infinity. Call tt the projection map from Z onto H\Z. 
For any y & H\Z choose z such that 'n:{z) = y, an open compact neighbour- 
hood Ky of z and denote by Uy the characteristic function of Ky. Consider 
the covering 

H\Z^ y n{Ky) 
yeH\z 
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Apply the first lemma. We get a covering 

H\Z=[jV, 

by open compact subsets, locally finite and for each j E J we can choose 
Uj E H \ Z such that Vj C 7:{Ky). Let (Kj)jgj be the partition of unity 
associated to the Vj. Put 

j 

This sum is locally finite and F is locally constant. The support condition 
is satisfied. For all z 



F{hz)drh 

H 

is strictly positive and locally constant. Thus we can take 

. _ F{z) 
JjjF{hz)drh 

Keep the notations of this last lemma. For / e S{Z) put 

(fiiTriz)) = / f{hz)drh 

JH 



Lemma 19.3 The map f ^ ip is a surjective map from S {Z) onto S{H\Z). 
The transpose map is a linear bijection from S'{H \ Z) onto the space of dis- 
tributions U on Z such that 

{UJ{hz))=AH{h){UJ{z)) 

The support of is contained into the projection of the support of F, hence 
is compact. To prove that ip is locally constant it is enough to consider the 
case where / is the characteristic function of some open compact suset K. 
Fix zq E Z and an open compact neighbourhood U oi zq. The action of H 
being proper, there exists a compact subset L oi H such that, for h E H the 
condition hUK imphes h E L. Thus, ior z E U 

ip{7i{z)) = J f{hz)drh 



105 



For z eU let 

L, = {h e L\hz e K}, M^ = {h e L\hz ^ K} 

They are both open and closed subsets of L. For each ho G L^^ there exists 
a neighbourhood Vh^ of Zq in U and a neighbourhood W^q of /i in L;^^ such 
that hz & K ior h & W^q and z & V^q. But L^^^ is compact so we can find a 
neighbourhood V of zq in U such that hz & K ior z & V and any e L^^. 
This means that L^,, G Lz ioi z E V. But we can argue in exactly the same 
way with Mz which, for z close enough to zq will give the inclusion Lz C . 
Hence Lz is "locally constant" and so is ip. Conversely let us start with 
(f e S{H \ Z). Fix u as in lemma B and put f{z) — (fi{7T{z))u{z); then / 
maps to ip. 

Now let S eS'{H\Z). We hft it to a distribution U on z hy 

{UJ) = {S, f f{h'z)drh') 

Jh 

If we replace f{z) by f{hz) 

[ f{hh'z)dr{h') = Anih) [ f{h'z)drh' 

so that 

(UJihz)) ^ AH{h){UJ{z)) 

Conversely if C/ is a distribution on Z satisfying the above condition we have 
to show that it is on the kernel of the map / i— > </?. So let / be such that 

/ f{hz)drh = 
Jh 

We may assume that / is not identically 0. Let u be as in Lemma A-3 and 
consider 

f {z)u{hz)drh 

Assume that z remains in the support of /, a compact subset of Z. Then 
because of the property of the support of u and the hypothesis that the action 
of H is proper the set K of all h such that, for some z, f{z)u{hz) 7^ is open 
and compact. Note that it is a neighbourhood of the neutral element of H. 
There exists an open compact subgroup Ki, contained in K and such that 

u{hhz)f{z) = u{hz)f{z), h eK,,he K, f{z) ^ 
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The subset X is a finite union of Xi— cosets: 

K = y Kih 

heKi\K 



Then 



and 



Then 



/ f{z)u{hz)drh = / f{z)u{hz)drh 
Jh Jk 



f{z)u{hz)drh^ J2 f{z)u{hz)Yol{Ki) 

" heKi\K 

(UJ) = {U, [ f{z)u{hz)drh) 
JH 

= ([/, f{z)<hz)Yo\{K^)) 

h£Ki\K 

= Vo\{K,){UJ{z)u{hz)) 

h€Ki\K 

{UJ{z)u{hz))drh 
{UJ{z)u{hz))drh 

H 

{U,f{h-h)u{z))AH{h)drh 



K 



H 



{U,f{hz)u{z))drh 

H 

Again if K' is the set of h such that, for some z, f{hz)u{z) ^ then K' 
is open and compact and contains an open compact subgroup K[ such that 
f{k'hz)u{z) = f{hz)u{z) for ki e K[, h e K', f{hz)u{z) ^ 0. replacing the 
integral by a finite sum we conclude that 

/ {U,f{hz)u{z))drh={U,u{z) [ f{hz)drh) = {U,u(p) 
Jh Jh 

which is if = 0. 

Now let us go back to the situation of Proposition A-1. Apply Lemma A- 
4 with X = G X X and H acting by {g,x) i-^ {gh~^,hx). Lemma A-4 



107 



provides a justification for the assertions concerning tlie map F ^ f and its 
transpose. Tliat U must be a tensor product d^®S follows from the equality 
5(G' X X) = S{G) ® S{X) and the fact that the left invariant distributions 
on G are the multiples of the Haar measure. The last computation in the 
proof is just a simple aplication of Fubini's theorem. 

Let X E X and let i/^ be its centralizer in H . A relatively invariant measure 
on H/Hx of multiplier x, a character of is a non zero measure Ux such 
that 

/ ^{sh)dux{h) = x{sr^ I ^{h)dux{h) 

J Hf Hx *^ Hf Hx 

Such a measure exists, and is essentially unique, if and only if 

We are interested in the case where x — ^g/ so that we must assume 
that A(3 = lS.Hx on restriction to H^. This is exactly the condition for the 
existence of an invariant measure on G/Hx- Also note that the stabilizer of 
a; in G is also Hx- 

We say that the orbit Hx of x in X carries a relatively invariant measure of 
multiplier x if? foi" ^ above 

/ \Lp{hx)\dux < +00, if e S{X) 

Similarly if /ix is an invariant measure on G/Hx we say that the orbit Gx 
carries an invariant measure if and only if 

/ \f{gx)\di,x{g)< +00, feS{Y) 
Jg/h^ 

Proposition 19.2 The orbit Hx carries a relatively invariant measure Vx of 

multiplier Ag/^h if o-nd only if the orbit Gx carries an invariant measure 
fix- Furthermore, if this is the case, we can normalize fix in such a way that 

fJ'X = 0{l'x) 

Suppose that Vx and fix exist as measures on H/Hx and G/Hx respectively. 
Then 

/ ^{gh)dvx{h)dii{g), ^eS{G/Hx) 

J G/H JH/H^ 
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is well defined and is in fact a left invariant measure. So we can normalize 
in such a way that 

/ ^g)dii^{g) = f / ^{gh)du^{h)dn{g) 

JG/H^ Jg/H JHIH^ 

Now if we start with </? e SiX) we can choose F e S{G x X) such that 

ip{x) = / F{g,x)dig 
Jg 

and then define / e S(Y) by 



f{gx)= / F(5f/i ^,hx)drh 
Jh 



If ^ we may choose F ^ 0, hence / ^ 0. Conversely if we start with 
/ we choose F and then compute ^p, both positive if / is positive. In this 
situation we have {9{S),f) = {S,ip). 
Suppose / ^ 0. then 



/ f {9x)diix{.g) = / F{gh ^ , hx)drhdiJ,^{g) 

Jg/H:, Jg/h^ Jh 

<p / F[gsh~^ ,hx)dj.hdvx{s)diJi{g) 

Jg/h Jh/h^ Jh 



— f / F{gh ^ ,hsx)drhdvx{s)diJi{g) 
Jg/h Jh/h^ Jh 

— f / / F(gh~^,hsx)drhdux{s)dijL{g) 
Jg/h Jh Jh/h^ 

= / [ [ F{gh'^, sx) ^"^.^l drh dv^{s)dn{g) 
Jg/h Jh Jh/h^ ^G[f^) 

^ f / / F{gh,sx)AG{g)drhdu^{s)diJ,{g) 

Jg/h Jh Jh/Hx 

^ F{g,sx)di/^{s)dig 
Jg Jh/Ho: 

-L 



<H/H^ 

ip{sx)du^{s) 



'H/H^ 

This shows that the first and last integrals are both finite or infinite, proving 
the first assertion of the proposition. The second one follows from the same 
computation with complex valued functions. 
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